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PREFACE 

During  the  past  twenty  years  there  have  been  many  controversies 
concerning  the  value  and  place  of  written  examinations.  There  have  also 
been  a  number  of  investigations  of  examinations  and  examination  "grades." 
Since  standardized  educational  tests  have  become  widely  used  a  number 
of  superintendents  and  teachers  have  proposed  that  they  replace  the 
written  examinations  set  by  teachers  and  other  school  officials.  More 
recently  some  attention  has  been  given  to  the  improvement  of  written  ex- 
aminations by  the  application  of  certain  principles  of  test  construction. 

Because  of  the  importance  of  the  written  examination  and  also  because 
a  number  of  inquiries  have  been  addressed  to  the  Bureau  of  Educational 
Research,  it  has  seemed  wise  to  organize  and  publish  a  summary  of  the 
important  ideas  relating  to  both  the  criticism  and  the  defense  of  examina- 
tions. To  this  there  have  been  added  a  number  of  suggestions  for  the  im- 
provement of  examinations.  It  is  hoped  that  this  bulletin  may  foster 
intelligent  thinking  relative  to  written  examinations  and  their  use  in  our 
schools. 

Although  this  bulletin  is  largely  the  product  of  the  labors  of  the 
Director  of  the  Bureau  of  Educational  Research  it  is  only  just  that  the 
contributions  of  other  members  of  the  staff  should  receive  recognition. 
Both  Mrs.  Charles  H.  Johnston  and  Mr.  Lloyd  B.  Souders  have  made 
substantial  contributions. 

Walter  S.  Monroe,  Director 


WRITTEN  EXAMINATIONS  AND  THEIR  IMPROVEMENT. 

INTRODUCTION 

Basis  of  a  Rational  Estimate  of  the  Value  of  Written  Examinations. 

Until  recently,  examinations  occupied  a  regular  place  in  the  work  of 
the  school.  Students  expected  them  as  a  matter  of  course,  and  the  ac- 
curacy of  the  marks  placed  upon  examination  papers  was  not  seriously 
questioned.  However,  for  a  number  of  years  written  examinations  set  by 
teachers  and  by  other  school  officials  have  been  subjected  to  criticism. 
During  this  period  the  defects  and  the  limitations  of  examinations  have  been 
thoroughly  canvassed.  Many  prominent  educators  have  advised  that  they 
be  abolished  entirely,  and  in  a  number  of  school  systems  this  has  act- 
ually been  done.  The  friends  of  examinations,  however,  have  urged 
their  merits  and  have  insisted  that  the  abolition  of  them  would  cause  our 
educational  system  to  deteriorate.  The  controversy  has  not  been  without 
prejudice  on  both  sides.  The  marking  of  examination  papers  involves 
much  drudgery  for  instructors.  Students  dislike  examinations  partly 
because  they  require  a  type  of  intensive  mental  activity  which  many  of 
them  prefer  to  avoid  and  partly  because  it  is  fashionable  in  many  schools 
to  oppose  them.  Conservatives,  naturally,  have  resented  any  proposal 
to  change  a  system  of  education  which  they  credited  with  producing  the 
educated  men  of  the  present  generation.  Some,  at  least,  have  expressed 
the  belief  that  examinations  have  been  largely  responsible  for  the  quality 
of  the  output  of  our  public  schools  and  colleges. 

In  evaluating  the  criticisms  and  the  defense  of  written  examinations 
it  is  imperative  that  one  keep  in  mind  the  fact  that  they  have  more  than 
one  function.  Written  examinations  are  not  merely  measuring  instru- 
ments, although  this  function  is  probably  most  prominent  in  the  thinking  of 
many  persons.  The  written  examination  is  used  as  an  instrument  for 
measuring  the  achievements  of  students,  but  it  also  affords  a  unique  type 
of  opportunity  for  learning.  Under  rather  well  defined  conditions,  certain 
tasks  are  set  for  the  pupil  and  he  is  required  to  demonstrate  within  a  limited 
time  what  he  is  able  to  do.  He  is  thrown  upon  his  own  resources  and  forced 
^  to  work  under  pressure.  In  the  actual  writing  of  his  answers  to  the  ques- 
tions of  the  examination  the  pupil  has  an  opportunity  to  learn.  Ideas 
tend  to  become  more  definite  as  a  result  of  expression  in  written  form. 
Frequently  the  pupil  gains  new  ideas  as  a  result  of  the  reflective  thinking 


he  does  in  answering  the  questions.  It  is  true  that  all  pupils  do  not  al- 
ways learn  in  taking  an  examination,  but  it  is  also  true  that  all  pupils  do 
not  take  advantage  of  all  other  educational  opportunities  which  are  offered 
them.  In  addition  to  the  actual  taking  of  the  examination,  the  pupil 
frequently,  as  a  preparation  for  it,  engages  in  review;  and,  because  he  knows 
that  later  he  must  take  the  examination,  he  has  a  stronger  motive  for  this 
review. 

Not  only  is  it  important  that  we  recognize  the  existence  of  functions 
other  than  the  one  of  measurement,  but  it  is  also  imperative  that  we  bear 
in  mind  two  distinctions.  First,  we  must  distinguish  between  criticisms 
of  examinations  and  criticisms  of  certain  kinds  of  examinations.  The 
fact  that  some  teachers  set  poor  examinations  does  not  furnish  an  adequate 
basis  for  concluding  that  all  examinations  should  be  abolished.  In  the 
second  place,  we  should  distinguish  carefully  between  criticisms  of  ex- 
aminations and  criticisms  of  the  ways  in  which  they  are  used.  Good  ex- 
aminations may  be  used  for  wrong  purposes.  For  example,  a  good  ex- 
amination might  be  given  to  a  pupil  or  a  group  of  pupils  merely  as  a  punish- 
ment for  some  misbehavior.  If  we  believe  that  such  use  is  not  justified 
it  does  not  follow  that  the  examination  itself  is  subject  to  adverse  criticism 
or  that  all  examinations  should  be  abolished. 


CHAPTER  I. 

CRITICISMS  OF  WRITTEN  EXAMINATIONS  SET  BY  TEACHERS 
AND  OTHER  SCHOOL  OFFICIALS 

The  arguments  advanced  for  and  against  examinations  have  dealt 
with  various  phases.  Some  of  the  criticisms  have  emphasized  the  effective- 
ness of  the  examination  as  a  measuring  instrument;  others  have  had  to  do 
with  the  purposes  for  which  examinations  are  used  by  teachers  and  by 
other  school  officials.  Some  criticisms  are  based  upon  facts,  while  others 
merely  represent  opinions.  In  the  following  pages  the  most  significant 
criticisms  have  been  summarized  and  grouped  under  a  few  major  heads. 
In  presenting  these  criticisms  there  will  be  no  attempt  to  point  out  their 
limitations  or  to  present  the  arguments  in  favor  of  written  examinations. 
This  will  be  reserved  until  the  second  chapter. 

1.  Examinations  yield  inaccurate  measures  of  achievement.  A 
number  of  criticisms  of  written  examinations  set  by  teachers  and  by  other 
school  officials  have  referred  to  their  effectiveness  as  instruments  for  meas- 
uring the  achievements  of  students.  These  criticisms  may  be  summarized 
under  six  heads. 

A.  Marking  of  examination  papers  subjective.  Scientific  investi- 
gation has  proved  that  the  marking  of  examination  papers  is  subjective, 
i.e.,  different  teachers,  when  working  independently,  tend  to  assign  widely 
varying  marks  to  the  same  paper.  An  investigation  by  Starch  and  Elliot1 
is  typical  of  many  that  have  been  made.  These  investigators  selected  a 
final  examination  paper  in  geometry,  written  by  a  student  in  one  of  the 
largest  high  schools  in  Wisconsin.  An  exact  reproduction  of  this  paper 
and  a  set  of  the  questions  were  sent  to  one  hundred  and  eighty  high  schools 
in  the  North  Central  Association.  It  was  requested  that  this  paper  be 
graded  according  to  the  practise  and  standards  of  the  school  by  the  princi- 
pal teacher  of  mathematics.  One  hundred  and  sixteen  acceptable  replies 
were  received.  The  papers  showed  evidence  of  having  been  marked  with 
unusual  care  and  attention.  In  seventy-three  schools  where  the  passing 
grade  was  75  the  lowest  mark  given  was  39  and  the  highest  88.  The  mode 
was  75,  with  twelve  teachers  giving  this  mark.     Of  the  one  hundred  and 


Starch,  Daniel,  and  Elliot,  E.  C.     "Reliability  of  grading  high  school  work  in  math- 
ematics,"  School  Review,  21:  254-59,  1913. 


sixteen  marks  assigned  to  this  paper,  two  were  above  90  and  one  was  below 
30.  Twenty  were  80  or  above  and  twenty  other  marks  were  below  60. 
Forty-seven  teachers  assigned  a  mark  passing  or  above,  but  sixty-nine 
teachers  thought  this  paper  not  worthy  of  a  passing  mark. 

Robert  L.  Morton2  reports  an  investigation  of  the  reliability  of  the 
marking  of  examination  papers  written  by  teachers  applying  for  a  license 
to  teach.  In  1904,  the  Ohio  Legislature  provided  for  uniform  questions 
for  the  teachers'  examination.  These  questions  were  to  be  prepared  in 
the  office  of  the  State  Superintendent  of  Public  Instruction  and  sent  to 
the  eighty-eight  county  boards  of  examiners.  Special  examiners  were 
appointed  in  each  county  to  rate  the  papers.  Morton  selected  an  arith- 
metic paper  from  the  files  of  one  board  of  examiners.  The  paper  was 
mimeographed,  care  being  taken  to  produce  exactly  the  language,  spelling, 
and  punctuation  of  the  original  paper.  A  copy  of  this  paper,  together  with 
the  questions,  was  sent  to  each  of  the  eighty-eight  county  superintendents 
in  Ohio  with  the  request  that  it  be  graded  by  the  special  examiner  for  arith- 
metic. Replies  were  received  from  fifty-five  counties.  The  lowest  mark 
given  to  the  paper  was  60  and  the  highest  99.  In  marking  the  answer 
given  to  one  question  on  this  paper  five  examiners  rated  it  at  zero,  twenty- 
one  at  10,  and  the  other  twenty-nine  assigned  marks  between  these 
extremes.  If  each  answer  had  been  rated  in  the  county  assigning  the  low- 
est mark  to  it,  the  total  "grade"  for  the  paper  would  have  been  28.  On 
the  other  hand,  if  the  highest  marks  assigned  to  the  answers  of  the  various 
questions  had  been  used  to  make  up  a  "grade",  a  mark  of  100  would  have 
been  given  to  the  paper.  Morton  investigated  in  a  similar  way  the  mark- 
ing of  a  paper  in  the  theory  and  practise  of  teaching  and  also  of  one  in 
geography.     Similar  variations  in  the  marks  were  found. 

^  A  striking  illustration  of  the  subjectivity  of  the  marking  of  exami- 
nation papers  by  college  instructors  is  cited  by  a  recent  writer3.  One  of 
the  group  of  expert  readers  assigned  to  the  marking  of  examination  papers 
in  history,  after  scoring  a  few  papers,  wrote  out  for  his  own  convenience 
what  he  considered  model  answers  to  the  questions.  By  some  mischance 
this  "model"  examination  paper  fell  into  the  hands  of  another  expert 
reader  who  graded  it  as  a  paper  written  by  a  student.  The  mark  he 
assigned  to  it  was  below  passing  and,  in  accordance  with  the  custom,  this 
"model"  was  rated  by  a  number  of  other  expert  readers  in  order  to  insure 
that  it  was  properly  marked.  The  marks  assigned  to  it  by  these  readers 
varied  from  40  to  90. 

2Morton,  Robert  L.  "The  examination  method  of  licensing  teachers,"  Educational 
Administration  and  Supervision,  6:  421,  November,  1920. 

3Wood,  Ben  D.  "Measurement  of  college  work,"  Educational  Administration  and 
Supervision,  7:  301-34,  September,  1921. 
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Scientific  investigation  of  the  marking  of  examination  papers  has 
been  sufficiently  extensive  to  prove  that,  except  in  a  very  few  instances, 
the  process  is  subjective.  Except  for  accidental  errors,  different  teachers 
should  assign  the  same  mark  to  an  examination  paper  in  spelling.  The 
marking  should  also  be  highly  objective  in  arithmetic  unless  there  is  an 
attempt  to  allow  partial  credit  for  examples  and  problems  partially  right 
or  for  correct  principle  when  the  answer  is  not  correct.  The  marking  of 
the  answers  to  questions  which  call  for  specific  facts,  such  as  dates,  names 
of  places,  or  persons,  should  approach  objectivity.  With  the  exception  of 
these  cases,  the  marking  as  it  is  ordinarily  done  is  highly  subjective,  and 
hence  the  "grades"  are  inaccurate  measures  of  achievement. 

As  might  be  expected,  the  degree  of  subjectivity  varies  with  differ- 
ent school  subjects.  It  is,  however,  sometimes  found  to  be  high  where  the 
nature  of  the  subject  matter  leads  one  to  expect  that  the  marking  will  be 
relatively  objective.  For  example,  Starch  and  Elliot  found  that  the  mark- 
ing of  an  examination  paper  in  geometry  was  just  as  subjective  as  one  in 
English  or  history.  Kelly4  found  that  the  rating  of  examination  papers 
in  algebra  was  considerably  more  objective  than   in  physics. 

B.  Questions  of  an  examination  not  equal  in  difficulty,  and  weighting 
by  teachers  subjective.  There  is  abundant  evidence  that  the  questions 
of  an  examination  are  generally  not  equal  in  difficulty.  Frequently,  in 
this  respect,  they  vary  widely.  When  the  questions  are  submitted  to  a 
large  number  of  pupils,  some  will  be  answered  correctly  by  a  large  percent 
of  the  pupils,  others  by  only  a  small  percent.  To  give  as  much  credit  for 
answering  an  easy  question  as  for  a  difficult  one  would  appear  to  introduce 
serious  errors  into  the  marks  assigned  to  the  papers.  Because  it  is  recog- 
nized that  the  questions  which  make  up  an  examination  are  generally  un- 
equal in  difficulty,  teachers  frequently  attempt  to  assign  appropriate 
weights.  For  example,  one  question  may  be  assigned  a  credit  of  15  points 
while  an  easy  one  is  given  a  credit  of  only  4  points.  One  investigation5 
has  shown  that  teachers'  estimates  of  the  difficulty  of  questions  are  highly 
subjective.  Twenty  teachers  were  asked  to  arrange  twenty-three  prob- 
lems in  arithmetic  in  the  order  of  their  difficulty.  A  very  wide  variation  in 
these  rankings  was  found.  One  problem  was  considered  the  easiest  by 
one  teacher  and  ranked  twenty-first  in  difficulty  by  another.  The  results 
of  this  investigation  seem  representative.     That  being  the  case,  any  weight- 


4Kelly,  F.  J.  "Teachers'  marking,"  Teachers  College,  Columbia  University,  Con- 
tributions to  Education,  No.  66,  1914. 

6Comin,  Robert.  "Teachers'  estimates  of  the  ability  of  pupils,"  School  and  Society, 
3:  67-70,  January  8,  1916. 
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Ing  of  questions  by  teachers  must  be  considered  highly  subjective,  and  hence 
not  a  satisfactory  corrective  for  the  unequal  difficulty  of  questions. 

C.  Content  of  examinations  not  in  agreement  with  educational  ob- 
jectives. The  criticism  is  frequently  made  that  teachers,  in  formulating 
examination  questions,  tend  to  ask  for  unimportant  details  and  to  neglect 
the  minimum  essentials  of  a  subject,  and  that,  therefore,  a  pupil's  per- 
formance on  an  examination  can  not  be  a  truthful  index  of  the  extent 
to  which  he  has  achieved  the  educational  objectives  set  for  him.  Some 
questions  are  described  as  "catch  questions."  By  this,  it  is  usually  meant 
that  such  questions  call  for  some  unimportant  detail  or  that  they  are  am- 
biguous in  some  way.  There  appears  to  have  been  no  scientific  investi- 
gation of  the  character  of  the  examination  questions  asked  of  pupils. 
However,  it  is  doubtless  true  that  this  criticism  has  justification  in  some 
cases  because  frequently  teachers  give  relatively  little  time  to  the  prepar- 
ation of  their  questions,  and  these  often  reflect  any  hobbies  or  prejudices 
which  the  teachers  may  have.  Experience  in  the  construction  of  stand- 
ardized educational  tests  has  shown  that  it  is  difficult  to  eliminate  all  am- 
biguity and  indefiniteness  in  questions.  Hence,  it  is  likely  true  that  many 
questions  are  not  well  stated,  and  for  this  reason  are  not  properly  under- 
stood by  those  taking  the  examination.  When  this  is  the  case,  the  "grades" 
tend  to  be  inaccurate  measures  of  achievement. 

When  an  examination  is  set  by  some  person  other  than  the  teacher 
of  the  class  it  not  infrequently  happens  that  many  of  the  questions  pertain 
to  topics  which  have  received  little  or  no  attention  during  the  instruction 
periods.  In  many  schools  it  seems  to  be  the  custom  for  the  superintendent 
or  the  principal,  without  consultation  with  the  teacher  in  charge,  to  make 
out  the  questions  for  the  final  examination  on  which  the  pupils'  semester 
grades  are  largely  based.  For  example,  in  a  fifth  grade  geography  class 
in  an  Illinois  city,  four  of  the  five  questions  of  the  examination  concerned 
current  conditions  about  which  the  children,  instructed  only  in  their  texts, 
knew  little.  A  few  pupils,  fortunate  enough  to  have  heard  these  matters 
-discussed  in  their  own  homes,  received  a  passing  grade.  The  majority  of 
the  class  failed.  This  examination,  interesting  and  in  itself  not  subject  to 
criticism,  should  not  have  been  used,  however,  as  a  means  for  measuring 
the  achievements  of  that  particular  class.  It  was  not  in  agreement  with 
the  educational  objectives  toward  which  the  teacher  had  directed  their 
efforts.  Such  examinations  are  "hard"  in  the  sense  that  capable  students 
will  answer  only  a  relatively  small  percent  of  the  questions  correctly,  and 
are  rightly  criticized  as  being  unjust  because  the  students  are  not  given 
an  opportunity  to  demonstrate  their  achievements. 
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D.  Rate  of  work  neglected.  The  usual  plan  is  to  set  an  examination 
which  practically  all  pupils  can  finish  in  the  time  allowed.  No  record  is 
kept  of  the  time  which  the  pupil  has  spent  in  writing  his  answers.  If  two 
pupils  write  papers  which  are  considered  equivalent  in  quality  but  one  has 
completed  the  examination  in  forty  minutes  and  the  other  in  ninety  min- 
utes, it  is  not  customary  to  distinguish  between  their  performances.  ^  Both 
will  receive  the  same  "grade."  This  means  that  the  rate  of  work  is  neg- 
lected. Since  the  rate  at  which  a  pupil  is  able  to  answer  questions  is  one 
index  of  his  ability,  the  ordinary  examination  fails  in  this  respect  to  secure 
a  truthful  measure  of  his  ability. 

E.  A  single  examination  does  not  offer  an  adequate  opportunity  for 
a  pupil  to  demonstrate  his  ability.  Some  critics  urge  that  a  single  exami- 
nation, even  when  carefully  prepared  and  graded,  will  not  in  general  yield 
a  reliable  measure  of  a  student's  ability.  McAndrew6,  in  reviewing  the 
work  of  the  New  York  City  high  schools,  says,  "New  York  City  high 
schools  use  them  (examinations)  in  deciding  the  promotion  to  the  training 
school  for  teachers.  We  have  every  year  some  students  whom  their 
teachers  have  complimented  regularly  but  who  fail  of  graduation  because 
of  a  three  hour  test  which  nullifies  the  work  of  four  years.  I  cannot  see 
how  a  pupil  writing  for  three  hours  can  be  tested  for  what  he  has  done  for 
a  year  or  more."  Courtis7  expresses  much  the  same  thought  in  the 
following  statement:  "The  best  examination  is  not  that  represented  by 
the  score  of  a  single  performance  in  a  single  day.  Human  effort  is  variable 
and  human  skill  too  easily  upset  to  make  it  fair  to  have  promotion  based 
upon  chance  scores." 

Thorndike8  has  summarized  a  number  of  investigations  carried  on  at 
Columbia  University  in  order  to  determine  the  reliability  of  the  "grades" 
made  on  college  entrance  examinations  as  a  basis  for  predicting  the  type 
of  work  which  the  student  will  do  in  college.  He  states  that  we  cannot 
estimate  the  success  of  the  student  in  college  from  his  grades  on  entrance 
examinations  with  "enough  accuracy  to  make  the  entrance  examinations 
worth  while  and  to  prevent  gross  injustice  being  done  to  any  individual. 
The  record  of  eleven  or  more  entrance  examinations  gives  a  less  accurate 
prophecy  of  what  a  student  will  do  in  the  latter  half  of  his  college  course 
than  does  his  high  school  record."     Similar  results  have  been  obtained  by 


•McAndrew,  Wm.  "Our  old  friend  the  examination,"  Proceedings  of  National  Edu- 
cational Association,  1916,  pp.  527-33.  >  . 

7Courtis,  S.  A.  "Standardizing  of  teachers'  examinations,  Proceedings  of  National 
Educational  Association,  1916,  pp.  1078-86.  _       _  n 

8Thorndike,  E.  L.  "The  future  of  the  college  entrance  examination  board,  Educa- 
tional Review,  31:  470-83,  May,  1906. 
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other  investigators.9  Studies  of  this  type  do  not  necessarily  prove  that 
examination  "grades"  are  inaccurate  measures  of  achievement.  Because 
of  other  elements  which  enter  into  college  life,  a  student  having  made  a 
satisfactory  record  in  his  secondary  school  may  not  carry  on  successfully 
his  work  in  college. 

F.  Marks  assigned  to  examination  papers  imply  subjective  norms. 
This  criticism  has  to  do  with  errors  in  interpreting  measures  of  achievement 
yielded  by  examinations  rather  than  with  errors  in  the  measures,  but  since 
examination  marks  are  usually  thought  of  as  measures,  it  may  be  con- 
sidered under  this  head.  In  reading  the  controversial  literature  on  written 
examinations  one  will  find  little  mention  of  the  criticism  expressed  by  the 
heading  of  this  paragraph,  but  in  the  illustrations  given  below  the  reader 
will  recognize  that  this  weakness  of  examination  marks  has  been  sensed  by 
most  pupils  and  teachers.  However,  they  appear  to  have  failed  to  analyze 
the  situation  sufficiently  to  grasp  the  source  of  the  difficulty.  In  the 
judgment  of  the  writer  this  is  one  of  the  most  serious  weaknesses  of  the 
traditional  examination. 

In  order  to  understand  how  norms  (standards)  are  used  in  connection 
with  the  grading  of  examination  papers  it  is  necessary  to  distinguish  be- 
tween scores,  or  measures,  and  "grades,"  or  marks.  A  score  simply  de- 
scribes the  performance  which  has  been  recorded  in  the  examination  paper. 
For  example,  a  pupil  may  answer  55  per  cent  of  the  questions  correctly. 
In  this  case  55  is  his  score.  If  a  certain  number  of  points  or  credits  had 
been  given  for  each  question  his  score  might  be  129  or  91,  or  217.  A 
"grade"  interprets  this  description  with  reference  to  certain  norms.  A 
"grade"  indirectly  describes  a  pupil's  performance  on  an  examination, 
but  it  also  tells  whether  the  pupil's  performance  is  to  be  considered  as  above 
passing  or  below  passing;  whether  he  is  to  receive  the  highest  mark  or  the 
lowest  mark  or  an  average  mark.  It  is  customary  to  describe  the  quality 
of  examination  papers  in  terms  of  the  percent  of  questions  answered  cor- 
rectly. For  example,  if  an  examination  includes  ten  questions  and  a  pupil 
answers  seven  of  them  correctly  and  an  eighth  one  partially  right,  he  is 
given  a  score  of  15  per  cent,  which  is  interpreted  to  mean  that  in  the  judg- 
ment of  the  examiner  he  has  answered  the  questions  75  percent  correctly. 
School  marks  or  "grades"  are  also  frequently  expressed  in  terms  of  per- 
cents.  Sometimes  they  are  expressed  in  terms  of  letters  or  other  symbols, 
but  these  in  turn  are  defined  in  terms  of  percents.  For  example,  the  grade 
of  "A"  may  be  defined  as  being  between  95  percent  and  100  percent. 

'Lincoln,  E.  A.     "Relative  standing  of  pupils  in  high  school  and  early  college,  and  their 
college  entrance  examinations,"  School  and  Society,  5:  417-20,  April  7,  1917. 
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Since  both  scores  and  "grades"  are  generally  expressed  in  terms  of 
percents,  it  is  only  natural  that  the  two  have  been  confused  and  that  scores 
have  been  used  as  "grades."  A  good  illustration  of  their  difference  came 
to  the  writer  recently.  An  examination  in  mathematics  was  given  to 
nearly  1000  freshmen  in  one  of  our  large  universities.  This  examination 
may  properly  be  described  as  "hard,"  considering  the  training  which  the 
students  had  received.  One  student  made  a  score  of  100.  The  lowest 
score  was  12.  The  average  was  approximately  55.  From  the  standpoint 
of  the  distribution  of  scores  this  was  a  "good  examination."  If  it  had 
been  easier,  so  that  any  considerable  number  of  pupils  received  scores  of  100 
percent,  it  would  have  been  defective.  If  it  had  been  so  "hard"  that  a 
considerable  number  of  students  made  zero  scores  it  would  also  have  been 
defective.  In  both  cases  it  would  have  failed  to  differentiate  between 
some  students  who  were  not  equal  in  ability.  However,  it  is  obvious  that 
if  a  passing  mark  of  70  or  15  is  adopted,  it  would  be  unjust  to  say  that  all 
pupils  having  scores  below  this  passing  mark  should  receive  a  "grade"  of 
failure.  The  passing  mark  for  this  particular  examination  should  be  in  the 
neighborhood  of  40.  If  in  this  case  it  is  desirable  to  have  the  scores 
represented  in  terms  of  "grades"  a  score  of  40  should  be  translated  into  a 
"grade"  of  70  or  whatever  passing  mark  this  institution  has  adopted. 

The  recognition  of  this  distinction  between  scores  and  "grades" 
enables  us  to  indicate  the  way  in  which  subjective  norms  are  implied  in 
"grades".  A  "grade"  is  not  a  pure  measure  or  description  of  the  pupil's 
performance.  It  is  rather  an  interpretation  of  the  measure  of  his  perform- 
ance with  reference  to  certain  norms.  When  no  distinction  is  made  and 
scores  are  used  as  "grades,"  pupils  will  receive  high  "grades"  if  the  ex- 
amination is  "easy;"  if  it  is  "hard"  they  will  receive  low  ones.  Thus,  the 
difficulty  of  the  examination  is  one  factor  in  establishing  the  norms  with 
reference  to  which  the  scores  are  interpreted  when  they  are  used  as  "grades". 
Severe  marking  will  tend  to  set  high  norms.  It  is  only  when  the  examina- 
tion is  of  average  or  "standard"  difficulty  and  the  marking  is  average  in 
severity  that  scores  and  "grades"  become  identical  in  magnitude.  Since 
the  norms  are  established  by  the  difficulty  of  the  examination  and  the 
severity  of  the  scoring,  they  must  be  subjective.  In  the  investigations  of 
the  marking  of  examination  papers  it  was  shown  that  teachers  varied  wide- 
ly in  their  judgments  concerning  the  worth  of  examination  papers.  There 
is  no  reason  to  expect  that  they  would  agree  more  closely  in  estimating 
the  difficulty  of  examinations.  Hence,  norms  which  depend  upon  teach- 
ers' estimates  of  which  questions  are  appropriate  for  examinations  and 
upon  their  marking  of  the  papers  must  be  considered  subjective. 
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2.     Undesirable   mental  processes  stimulated  by  examinations.     A 

number  of  critics  have  urged  that  examinations — and  especially  the  prepar- 
ation for  them — tend  to  stimulate  undesirable  mental  processes,  and  that 
as  a  result  a  student's  learning  is  not  of  the  right  kind.  It  is  claimed  that 
memory  is  emphasized  to  the  exclusion  of  higher  mental  processes.  Many 
questions  call  only  for  facts,  and  even  in  the  case  of  those  which  require 
reasoning  the  student  must  have  some  facts  with  which  to  reason.  He  is 
generally  asked  to  answer  the  questions  without  having  access  to  his  text 
books  or  to  other  sources  of  information.  It  is  seldom  that  the  examina- 
tion provides  the  student  with  any  of  the  facts  or  principles  which  he  re- 
quires in  the  reasoning  process.  Hence,  what  he  records  upon  his  ex- 
amination paper  must  be  based  upon  his  own  store  of  information.  Thus, 
it  is  undoubtedly  true  that  examinations  make  heavy  demands  upon  the 
memory  of  the  pupils. 

Those  who  have  supported  this  criticism  of  examinations  insist  that 
this  emphasis  upon  memory  tends  to  change  the  child's  mind  into  an 
automatic  machine.  The  following  statements  are  typical:  "It  is  mere 
commonplace  to  say  that  in  many  schools  the  best  preparation  for  an  ex- 
amination is  to  have  in  memory  a  vast  number  of  details.  The  student  may 
undertake  the  examination  in  such  a  condition  of  brain  fatigue  that  he 
would  find  it  difficult  to  solve  a  simple  original  problem;  but  if  he  has  this 
plethoric  memory  of  details  he  will  succeed."10  "The  memory  which  gives 
back  knowledge  in  the  exact  form  in  which  it  is  received  is  likely  to  shine 
at  examinations."11 

In  this  connection  it  is  claimed  examinations  place  a  premium  upon 
"cramming."  Since  memory  is  emphasized  it  is  only  natural  that  stu- 
dents should  engage  in  an  intensive  review  of  the  course  immediately 
preceding  the  examination  period.  It  is  a  well  established  principle  that 
the  recency  of  an  experience  materially  affects  its  recall.  The  critics  of 
examinations  insist  that  students  who  have  given  little  attention  to  the 
work  of  the  course  during  the  term  may  write  a  creditable  examination 
paper  merely  as  a  result  of  an  intensive  cramming  immediately  preceding 
the  examination  date.12  Since  this  is  true,  students  tend  to  loaf  during 
the  term  and  then  "cram"  for  examinations. 

When  a  student's  learning  is  limited  to  a  brief  period  of  intensive 
cramming  immediately  preceding  the  examination,  he  will  tend  to  forget 
very  rapidly.     In  addition,  he  has  missed  the  regular  growth  that  was 

10Monroe,  Paul.     Encyclopedia  of  Education,  Vol.  2,  536-38. 

"Schaeffer,  N.  C.     "The  unmeasurable  in  teaching,"  N.  E.  A.  Proceedings,  51:   169- 
72,  1913. 

"Hollister,  H.  A.   High  School  Administration.   Boston:  D.  C.  Heath  &  Company,  1909. 

16 


possible  during  the  term  and  has  had  no  opportunity  to  organize  and  re- 
late the  various  items  of  the  course. 

3.  Examinations  tend  to  become   educational  objectives.     It  has 

been  charged  that  teachers  tend  to  emphasize  examinations  in  such  a  way 
that  the  students  come  to  think  of  them  as  the  educational  objectives  to  be 
attained.  Because  the  fear  of  making  a  low  "grade"  on  an  examination 
is  an  effective  motive,  teachers  frequently  use  it.  The  students  are  re- 
minded of  approaching  examinations  and  are  warned  that  they  will  not  be 
able  to  pass  them  unless  they  study  certain  topics.  When  the  "grade" 
received  upon  the  final  examination  determines  the  student's  success  in 
the  course,  it  naturally  follows  that  he  will  think  of  "passing  the  examina- 
tion" as  the  objective  to  be  attained.  Sometimes  students  secure  former 
examination  questions  or  compile  a  list  of  the  topics  the  instructor  has 
emphasized  and  study  these  to  the  exclusion  of  other  topics  which  may  be 
more  important.     Thus,  the  real  objectives  to  be  attained  are  obscured. 

4.  Examinations  injurious  to  health  of  students.  Because  of  the 
strenuous  preparation  for  examinations  and  also  because  of  the  obvious 
strain  which  accompanies  the  taking  of  an  examination,  a  number  of  critics 
have  urged  that  examinations  are  injurious  to  the  health  of  students. 
Even  students  who  have  been  faithful  during  the  term  will  frequently 
cram  for  the  examination  because  of  a  desire  to  take  a  high  rank  in  their 
class  or  because  of  the  additional  credit  for  quality  that  is  given  in  some 
institutions.  When  a  student  is  required  to  write  for  a  number  of  hours 
during  a  single  day,  as  sometimes  occurs,  it  is  obvious  that  even  under  the 
best  conditions  he  has  expended  a  large  amount  of  nervous  energy. 

5.  Time  devoted  to  marking  of  examination  papers  might  be  more 
profitably  employed.  A  majority  of  our  larger  colleges  and  universities 
have  recognized  the  fact  that  the  marking  of  ordinary  examination  papers 
makes  heavy  demands  upon  the  time  of  instructors  and  have  provided 
"assistants"  or  "readers"  who  are  to  rate  the  papers.  Not  infrequently 
the  examination  period  is  dreaded  by  instructors  as  well  as  by  students 
because  of  the  drudgery  involved  in  reading  examination  papers.  One 
writer13  has  estimated  that  it  requires  approximatly  three  hours  to  con- 
struct, give,  and  mark  an  examination  for  an  average  class.  A  teacher 
will  give  twenty  or  more  examinations  per  year.  This  means  that  each 
teacher  devotes  not  less  than  sixty  hours  to  the  construction,  giving,  and 
marking  of  examinations.  This  estimate  is  conservative.  Many  teachers 
spend  much  more  time  than  this.  The  critics  of  examinations  have  pointed 
out  that  this  time  might  be  more  profitably  devoted  to  other  school  activities. 

13McCaIl,  W.  A.     "New  kind  of  school  examination,"  Journal  of  Educational  Research, 
1:33-46,  January,  1920. 
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CHAPTER  II. 

THE  DEFENSE  OF  WRITTEN  EXAMINATIONS  SET  BY  TEACHERS 
AND  OTHER  SCHOOL  OFFICIALS 

The  criticisms  of  written  examinations  summarized  in  the  preceding 
chapter  appear  to  make  a  strong  case  against  their  use  in  schools.  Some 
of  the  criticisms  are  established  facts,  and  others  corroborate  our  general 
observations.  It  is,  however,  necessary  to  remember  that  only  one  side 
of  the  case  has  been  presented.  Some  of  the  criticisms  are  much  less 
serious  than  their  advocates  would  have  us  believe.  Others  are  not  funda- 
mentally criticisms  of  examinations,  but  rather  of  the  way  in  which  they 
are  used  and  of  other  phases  of  our  school  procedure.  In  addition,  certain 
significant  merits  of  examinations  were  not  mentioned.  In  this  chapter 
the  criticisms  set  forth  in  Chapter  I  will  be  examined,  and  the  defense  of 
examinations  presented.  In  Chapter  III  we  shall  explain  certain  methods 
of  improving  written  examinations  which  tend  to  eliminate  or  greatly 
minimize  certain  defects  that  now  exist. 

1.  Measurement  of  abilities  of  students  necessary  to  high  degree  of 
school  efficiency.  In  considering  written  examinations  as  measuring  instru- 
ments, it  is  necessary  to  bear  in  mind  that  the  work  of  the  school  cannot  be 
carried  on  in  an  efficient  manner  without  from  time  to  time  measuring  the 
abilities  of  students.  These  measurements  are  essential  to  the  organization 
and  administration  of  our  schools  as  well  as  to  the  instruction  of  the  class 
room.  The  promotion  and  classification  of  pupils,  the  guidance  of  pupils, 
both  educational  and  vocational,  the  supervision  of  instruction,  and  even  in- 
struction itself,  cannot  be  most  efficient  unless  the  abilities  of  the  pupils 
are  measured  at  intervals.  If  these  measurements  are  not  made  in  one 
way  they  must  be  in  another.  If  written  examinations  are  abolished  the 
abilities  of  pupils  must  be  measured  in  other  ways,  or  the  efficiency  of  our 
schools  will  decrease. 

2.  Substitutes  tor  written  examinations.  It  has  been  proposed  by 
some  critics  that  substitutes  for  written  examinations  may  be  used. 
Standardized  educational  tests,  teachers'  estimates,  and  daily  "grades" 
furnish  types  of  information  upon  which  the  measurement  of  the  pupil's 
achievement  may  be  based.  However,  it  would  seem,  for  the  following 
reasons,  that  such  means  of  measuring  cannot  be  considered  as  satisfac- 
tory substitutes  for  written  examinations. 
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A.  Standardized  educational  tests  versus  examinations.  Standard- 
ized educational  tests  have  been  shown  to  be  superior  to  ordinary  examina- 
tions as  measuring  instruments,  but  in  considering  the  proposal  that  they 
replace  written  examinations  certain  facts  must  not  be  overlooked.  The 
number  of  satisfactory  standardized  educational  tests  is  as  yet  very 
limited.  In  only  a  few  school  subjects,  such  as  handwriting,  spelling, 
arithmetic,  and  oral  and  silent  reading,  do  we  have  available  standardized 
educational  tests  which  might  be  used  as  substitutes  for  written  examina- 
tions set  by  the  teacher.  Even  in  these  subjects  it  may  happen  that  for 
some  reason  the  teacher  has  emphasized  certain  topics  and  omitted  or 
treated  casually  others.  Furthermore,  official  courses  of  study  vary 
widely  from  city  to  city.  Standardized  educational  tests  are  of  necessity 
confined  to  those  topics  which  are  uniformly  taught  or,  at  most,  to  those 
topics  which  are  generally  taught;  therefore,  in  a  particular  instance  there 
may  not  be  available  any  standardized  educational  test  which  is  adapted 
to  the  instruction  which  the  class  has  been  receiving.  A  teacher,  however, 
can  construct  an  examination  which  is  specifically  designed  to  measure  the 
results  of  instruction  given  to  a  particular  class.  Thus,  standardized 
educational  tests  can  be  used  only  to  a  limited  extent  as  substitutes  for 
examinations  set  by  the  teacher  or  by  other  school  officials. 

In  this  connection  it  is  well  to  remember  that  standardized  educa- 
tional tests  are  not  perfect  measuring  instruments.  Even  the  best  of  them 
do  not  begin  to  approach  the  degree  of  accuracy  to  which  we  are  accus- 
tomed in  the  measurement  of  physical  objects.  The  results  which  they 
yield  may  involve  errors  so  large  as  to  distort  or  make  entirely  erroneous 
many  of  our  interpretations.  Thus,  in  considering  the  errors  which  are 
involved  in  examination  "grades"  we  should  not  forget  that  our  best  in- 
struments for  measuring  mental  abilities  are  far  from  perfect  with  refer- 
ence to  accuracy. 

B.  Teachers'  estimates  versus  examination  marks.  Some  of  the 
critics  of  examinations  have  insisted  that  experienced  teachers  are  able  to 
estimate  the  achievements  of  students  by  reason  of  their  acquaintance 
with  them  during  the  term.  It  has  been  claimed  that  these  estimates  will 
be  more  truthful  measures  of  achievement  than  the  "grades"  obtained  from 
a  written  examination  given  at  the  end  of  the  term.  There  is  no  doubt 
that  experienced  teachers  can,  under  favorable  circumstances,  estimate 
with  considerable  accuracy  the  achievements  of  students.  If  the  class 
is  reasonably  small  and  if  the  teacher  has  used  methods  of  instruction  which 
have  called  for  frequent  oral  and  written  performances  by  the  students 
and  has  kept  a  careful  record  of  the  quality  of  these  performances  through- 
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out  the  term,  the  estimates  may  be  relatively  accurate  measures  of  the 
achievements  of  the  students.  However,  there  are  certain  limitations 
which  should  be  noted. 

Teachers  may  be  unduly  influenced  in  their  estimates  by  the  recent 
performances  of  the  students.  Unless  careful  records  have  been  kept 
throughout  the  term,  inferior  performances  at  the  beginning  tend  to  be 
overshadowed  and  the  teacher's  final  estimate  based  upon  the  work  of  the 
last  few  weeks.  In  case  the  class  is  a  large  one,  the  teacher  does  not  have 
an  adequate  opportunity  of  becoming  intimately  acquainted  with  the 
students.  Teachers'  estimates  are  likely  to  be  materially  affected  by  per- 
sonal characteristics  of  pupils.  A  pupil  with  a  pleasing  personality  or  one 
who  is  liked  by  the  teacher  is  likely  to  be  rated  higher  than  one  who  is 
unattractive. 

If  the  class  work  is  conducted  so  that  there  is  little  or  no  written  per- 
formance on  the  part  of  the  students,  teachers'  estimates  will  necessarily 
be  based  almost  wholly  upon  the  oral  responses  which  students  give  during 
the  class  period.  Some  pupils  make  a  good  showing  in  class  when  the 
recitation  is  oral,  but  are  at  a  decided  disadvantage  when  asked  to  record 
their  answers  in  writing.  Frequently  this  difficulty  is  due  to  the  fact  that 
they  are  careless  in  their  thinking  and  do  not  have  well  formed  ideas  to 
express.  In  oral  recitation  they  are  able  to  make  a  fair  showing  because 
of  personal  characteristics  and  because  of  the  stimulus  of  detailed  ques- 
tioning by  the  instructor.  Furthermore,  in  a  class  discussion  a  bright 
student  who  has  a  good  command  of  language  may  easily  pick  up  ideas 
from  other  members  of  the  class  and  recall  items  from  his  general  experi- 
ence sufficient  to  make  a  good  showing.  On  the  other  hand,  there  are  stu- 
dents who  express  themselves  most  effectively  in  writing.  They  may 
be  good  thinkers  but  a  little  slow  in  their  mental  processes  and  not  clever 
in  discussion.  Thus,  it  is  difficult  or  impossible  for  teachers  to  estimate 
accurately  the  real  achievements  of  students  from  oral  recitations  alone. 
In  classes  where  a  large  amount  of  written  work  is  required,  as  in  English 
composition,  the  teacher  has  an  opportunity,  in  formulating  estimates, 
to  consider  the  pupil's  written  as  well  as  oral  performances.  In  such  sub- 
jects, the  proposal  to  replace  written  examinations  by  teachers'  estimates 
based  upon  the  work  of  the  course  has  much  more  merit  than  in  other 
subjects  where  there  has  been  only  a  small  amount  of  written  work.  But 
even  when  much  daily  written  work  is  required,  a  written  examination 
covering  the  work  of  the  term  may  give  additional  information  concerning 
the  real  achievements  of  some  students. 
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C.  Daily  "grades"  versus  examination  marks.  It  has  been  pro- 
posed that  the  best  measure  of  a  student's  achievements  throughout  a 
term  is  furnished  by  the  average  of  his  daily  "grades."  These  "grades" 
include  teachers'  estimates  of  pupils'  performances  in  class,  marks  given 
for  written  work  submitted,  and  "grades"  earned  on  short  quizzes.  Those 
who  favor  using  the  average  of  these  marks  as  a  pupil's  final  grade  point 
out  that  by  this  means  we  secure  a  measure  of  a  pupil's  achievement  which 
depends  upon  the  quality  of  his  work  day  by  day  and  which,  furthermore, 
is  the  average  of  a  large  number  of  measurements.  Probably  no  one 
would  contend  that  a  student's  daily  "grades"  should  not  receive  con- 
sideration in  determining  his  final  standing.  There  are,  however,  certain 
dangers  in  taking  the  average  of  a  pupil's  daily  "grades"  as  his  final  mark. 
In  the  first  place,  if  the  student  knows  that  his  final  mark  depends  only 
upon  his  daily  work  he  is  likely  to  study  for  the  day  alone.  His  daily 
"grade"  is  based  upon  a  rather  small  unit  of  work.  Furthermore,  it  is 
based  upon  his  performance  immediately  after  studying  the  assignment. 
Thus,  a  pupil's  ability  to  organize  the  content  of  a  course  and  his  retention 
of  it  are  likely  to  receive  little  consideration  in  making  up  his  final  mark. 
Of  course,  it  is  possible  for  a  teacher  to  ask  the  student  to  review  frequently 
and  to  summarize  and  organize  the  work  of  the  course  at  the  end  of  the 
term.  However,  it  is  difficult  to  convince  students  of  the  necessity  of 
reviewing  the  work  of  a  course  if  the  performance  on  this  review  receives 
no  more  weight  in  determining  the  final  "grades"  than  the  performances 
during  an  equal  period  of  time  elsewhere  in  the  course.  When  classes 
are  large  it  is  difficult  to  have  each  student  recite  each  day,  at  least  to  a 
sufficient  extent  to  furnish  any  basis  for  assigning  a  daily  "grade."  When 
the  members  of  the  class  are  called  upon  only  occasionally  it  frequently 
happens  that  they  prepare  only  those  lessons  upon  which  they  think  they 
will  be  questioned.  Of  course,  a  resourceful  instructor  can  do  much  to 
discourage  this  practise  but  the  fact  remains  that  daily  "grades"  may  not 
be  representative  samples  of  a  student's  achievements  in  a  course.  A  writ- 
ten examination  covering  the  work  of  the  entire  course  or  of  a  large  division 
of  it  will  furnish  a  measure  of  achievement  which  can  be  secured  in  no  other 
way. 

Conclusion:  No  satisfactory  substitute  for  written  examinations. 
Teachers'  estimates  and  daily  grades  furnish  certain  types  of  measures 
of  the  achievements  of  pupils.  In  certain  cases  standardized  educational 
tests  can  be  used  to  secure  relatively  accurate  measures  of  achievement  in 
certain  fields.  However,  these  means  of  measuring  cannot  be  considered 
satisfactory  substitutes  for  written  examinations.     The  written  examina- 
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tion  yields  a  type  of  achievement  which  cannot  be  secured  through  any 
other  means.  This  measurement  is  more  important  for  some  school 
subjects  than  for  others  but  in  relatively  few  is  one  justified  in  abolishing 
the  written  examination  as  a  means  of  measurement. 

3.  Inaccuracy  of  examination  marks.  In  Chapter  I  several  sources 
of  error  in  examination  marks  were  mentioned.  In  the  case  of  some  of 
these  sources  sufficient  experimental  evidence  was  presented  to  be  con- 
vincing. The  only  way  in  which  these  sources  may  be  eliminated  or 
minimized  is  by  modifying  examinations.  These  modifications  will  be 
treated  in  Chapter  III,  but  three  sources  of  error  may  be  commented  on  at 
this  place. 

A.  Neglect  of  the  rate  of  work  not  necessary.  It  is  not  necessary 
that  the  rate  of  work  be  neglected  in  examinations  set  by  teachers  and 
other  school  officials.  It  is  easily  possible  to  take  into  account  a  student's 
rate  of  work  in  determining  the  mark  which  he  is  to  receive  on  his  examina- 
tion paper.  One  plan  is  to  set  an  examination  of  sufficient  length  to  keep 
all  members  of  the  class  employed  during  the  entire  period.  In  marking 
the  papers,  the  number  of  questions  which  the  student  has  answered  should 
be  considered  as  well  as  the  quality  of  his  answers.  Another  procedure 
is  to  have  each  student  record  the  time  when  he  finishes.  This  record  will 
enable  the  teacher  to  determine  the  amount  of  time  devoted  to  answering 
the  questions.  In  the  measurement  of  handwriting  and  of  silent  reading, 
as  well  as  of  abilities  in  a  number  of  other  fields,  it  is  extremely  important 
that  the  rate  of  work  be  considered. 

B.  Unequal  difficulty  of  questions  not  a  serious  defect.  It  does  not 
appear  that  accurate  measurements  of  the  abilities  of  students  can  be 
secured  by  giving  the  same  credit  for  answering  an  easy  question  as  for 
answering  a  difficult  one.  However,  investigations  of  this  question  in 
connection  with  the  scoring  of  standardized  educational  tests  have  in- 
dicated that  the  errors  introduced  by  this  procedure,  which  appears  to  be 
illogical,  are  not  large.  After  having  weighted  the  exercises  of  his  language 
and  grammar  tests  on  the  basis  of  difficulty,  Charters1  dropped  the 
weights  because  he  found  that  the  correlation  between  the  weighted  and 
unweighted  scores  was  slightly  over  0.90.  A  number  of  other  test  makers 
have  likewise  used  exercises  which  were  unequal  in  difficulty  without 
assigning  any  weighted  credits  to  them.  A  number  of  other  tests  which 
consist  of  exercises  arranged  in  ascending  order  of  difficulty  have  been 


Charters,  W.  W.     "Constructing  a  language  and  grammar  scale,"Journal  of  Educa- 
tional Research,  1:249-58,  April,  1920. 
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scored  by  taking  the  number  of  exercises  done  correctly,  which  amounts  to 
giving  as  much  credit  for  doing  an  easy  exercise  as  for  doing  a  more  difficult 
one. 

The  writer2  has  shown  in  the  case  of  certain  standardized  educational 
tests  that  the  error  introduced  by  disregarding  the  inequality  of  the  ques- 
tions in  scoring  is  not  larger  than  the  variable  error  of  measurement  due  to 
the  variability  of  the  performances  of  pupils.  It,  therefore,  seems  likely 
that  the  error  introduced  by  disregarding  the  unequal  difficulty  of  the 
questions  of  an  examination  will  not  be  significant  in  most  cases.  When 
the  examination  consists  of  as  many  as  twenty  or  more  questions  the  un- 
equal difficulty  of  the  questions  probably  can  be  safely  disregarded.  Of 
course,  it  is  wise  to  avoid  extreme  variations  in  difficulty  whenever  possible. 
Those  setting  examinations  should  endeavor  to  formulate  questions  which 
do  not  differ  widely  in  this  respect. 

There  is  some  doubt  whether  the  difficulty  of  a  question  can  furnish  a 
criterion  for  determining  the  amount  of  credit  which  should  be  given  for 
answering  it  correctly.  Achievements  are  not  valuable  merely  because 
they  are  difficult  but  because  of  the  relation  which  they  sustain  to  our 
educational  objectives.  Questions  which  are  relatively  unimportant  may 
be  very  difficult.  In  fact,  lack  of  instruction  is  one  reason  why  a  question 
is  difficult.  Therefore,  topics  which  are  assigned  minor  places  in  a  course 
of  study  may  furnish  the  basis  for  very  difficult  questions,  while  the  topics 
which  have  been  emphasized  because  of  their  importance  may  furnish  the 
easier  questions.  It  is  not  unlikely  that  the  best  basis  for  weighting  ques- 
tions is  their  social  importance  rather  than  their  difficulty. 

C.  Inaccuracy  of  single  examination.  It  has  been  urged  that  a 
single  examination  cannot  yield  an  accurate  measure  of  the  achievements 
of  students.  It  is  human  nature  to  be  variable  in  one's  performances. 
For  certain  reasons  students  whose  achievements  are  ordinarily  high  may 
make  low  "grades"  on  a  single  examination.  It  must  therefore  be 
admitted  that  many  of  the  "grades"  yielded  by  a  single  examina- 
tion are  likely  to  involve  serious  errors.  However,  in  considering  the 
significance  of  these  errors  with  reference  to  the  use  of  written  examina- 
tions, it  is  necessary  to  distinguish  between  examination  "grades"  and 
final  "grades."  The  fact  that  the  "grade"  for  a  single  examination  may  be 
inaccurate  and,  hence,  that  a  final  "grade"  should  not  be  determined  by  a 
single  final  examination  is  not  sufficient  reason  for  abolishing  written  ex- 


2Monroe,  Walter  S.     "A  discussion  of  the  performances  of  pupils  on  exercises  of  varying 
difficulty,"  School  and  Society,  15:341-43,  March,  1922. 
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animations.  It  simply  means  that  more  than  one  written  examination 
should  be  given  or  that  other  measures  of  achievement  should  be  taken 
into  consideration  in  determining  final  "grades."  Teachers'  estimates 
and  daily  grades  should  have  a  place.  Any  written  tests  which  may  be 
given  during  the  term  should  also  be  given  some  weight.  A  final  exam- 
ination, however,  deserves  a  place  also.  In  some  subjects  it  should  be 
given  more  weight  than  in  others. 

4.  Examinations  force  students  to  review  and  organize  content  of 
course.  One  of  the  criticisms  made  in  Chapter  I  was  that  written  exami- 
nations tended  to  emphasize  memory  and  consequently  to  minimize  the 
opportunities  for  the  functioning  of  the  higher  mental  processes.  On  the 
other  hand,  the  friends  of  written  examinations  insist  that  they  tend  to 
produce  exactly  the  opposite  effect.  They  contend  that  final  examina- 
tions have  a  positive  value  in  that  they  tend  to  force  students  to  review 
and  to  organize  the  content  of  a  course.  This  is  a  very  important  phase 
of  learning  in  many  subjects.  It  is  pointed  out  that  the  emphasis  which 
examinations  place  upon  memory  is  not  undesirable.  Students  should 
learn  many  facts,  and  they  should  learn  them  with  sufficient  accuracy  and 
in  sufficient  detail  to  answer  most  examination  questions.  The  memoriza- 
tion of  a  considerable  number  of  facts  is  to  be  commended.  Even  the  in- 
tensive cramming  which  is  deplored  by  many  persons  has  educational 
value.  If  a  student  studies  throughout  the  course,  an  intensive  review 
and  organization  of  the  material  is  extremely  valuable.  In  fact,  the  profit 
to  the  student  is  probably  relatively  greater  for  time  expended  in  this  work 
than  for  any  other  equal  amount  of  time  devoted  to  the  subject.  Unless 
carried  to  an  extreme,  cramming  is  undesirable  only  when  it  has  not 
been  preceded  by  thoughtful  study.  Even  in  this  case  it  is  better  to  have 
the  student  "cram"  for  the  examination  than  to  go  through  the  course 
without  engaging  in  any  learning.  Incidentally,  it  should  be  noted  that 
the  existence  of  "cramming"  is  not  the  fault  of  the  examinations  but  of 
the  type  of  instruction  given  and  of  the  policy  of  making  the  student's 
final  grade  depend  wholly  or  very  largely  upon  his  examination  mark. 
With  proper  instruction  and  with  considerable  weight  given  to  his  record 
throughout  the  course,  it  will  not  be  possible  for  a  student  to  loaf  during 
the  term  and  secure  a  creditable  final  "grade"  by  "cramming"  immediately 
preceding  the  examination.  The  fact  that  some  students  are  able  to  earn 
high  marks  merely  by  a  brief  period  of  intensive  study  immediately  pre- 
ceding the  examination  is  more  of  a  reflection  upon  the  content  of  the 
course  than  upon  examinations.  Undoubtedly,  there  are  some  students 
who  are  sufficiently  brilliant  to  learn  within  a  period  of  a  few  days  as  much 
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as  the  average  student  learns  throughout  the  course;  and,  furthermore, 
it  is  probably  true  that  in  some  of  our  courses  relatively  little  is  demanded 
of  students. 

It  may  be  pointed  out,  also,  that  the  extreme  emphasis  upon  memory 
cited  by  some  critics  is  not  the  fault  of  examinations  in  general  but  of 
particular  types  of  questions.  It  is  possible  to  ask  questions  which  may 
be  answered  entirely  from  memory.  It  is  also  possible  to  ask  questions 
which  require  much  reasoning  and  organization  of  information. 

The  writing  of  an  examination  may  itself  be  an  important  part  of  the 
student's  learning.  "There  is  no  impression  without  expression."  The 
writing  of  a  three  hour  examination  is  undoubtedly  an  intensive  form  of 
expression.  In  case  the  questions  are  such  that  the  reasoning  and  organi- 
zation of  information  are  required,  the  student  may  learn  a  great  deal 
from  the  act  of  writing  his  answers  to  the  questions.  From  this  point 
of  view,  it  may  be  urged  that  a  student  who  is  not  required  to  take  final 
examinations  is  deprived  of  an  important  opportunity  for  learning.  Not 
infrequently  students  who  have  been  "excused  from  examinations"  in 
high  school  report  that  they  experience  a  distinct  handicap  when  they 
become  college  students. 

5.  Examinations  furnish  effective  motive.  Although  the  written  ex- 
amination should  not  be  defended  merely  for  the  reason  that  it  fur- 
nishes an  effective  motive,  it  must  be  admitted  that  this  motive  makes  a 
special  appeal  to  many  pupils.  In  so  far  as  possible,  the  pupils  should  be 
motivated  by  the  intrinsic  values  of  the  subject  matter.  However,  these 
values  are  frequently  remote  from  the  life  of  the  student,  and  it  is  necessary 
to  resort  to  motives  which  make  a  more  immediate  appeal.  This  point  of 
view  is  effectively  set  forth  in  the  following  quotation:  "Something  in 
addition  to  merely  appealing  to  the  future  motives  seems  necessary  at 
times  to  urge  along  the  average  reader,  some  means  whereby  his  future 
means  are  made  comparatively  more  immediate.  Not  merely  does  the  ex- 
amination stimulate  the  reader  generally,  but  it  also  helps  to  determine  his 
specific  interest  and  behavior.  If  the  child  feels  himself  compelled  to  give 
an  account  of  what  comes  within  the  field  of  his  perception,  he  will  soon  de- 
velop a  habit  of  attending  to  that  for  which  he  will  probably  be  held  ac- 
countable. The  examination,  when  properly  applied,  constantly  re- 
minds the  reader  that  he  is  responsible  for  the  mastery  of  what  is  proposed 
for  his  learning.'/3 

3Myers,  G.   C.     "Examinations   and   the   learner,"   Educational   Review,   54:274-84, 
June,  1917. 
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6.  Proper  use  of  examinations.  As  we  have  just  pointed  out,  ex- 
aminations may  fulfill  a  very  important  function  in  the  education  of 
students.  Also  they  may  be  abused.  In  some  schools  final  examinations 
are  used  as  a  punishment.  Students  who  attain  a  fixed  average  in  their 
daily  grades  and  who  have  not  fallen  below  a  minimum  in  their  deportment 
are  excused  from  final  examinations.  In  such  a  case  the  examination  is 
regarded  both  by  teachers  and  students  as  undesirable  or  evil  and  natur- 
ally fails  to  fulfill  its  function.  This  practise  of  excusing  some  students 
from  examinations  is  not  a  defensible  one.  The  implication  is  that  the 
examination  is  not  an  integral  part  of  the  school  work,  but  merely  a  pun- 
ishment or  a  last  resort  measuring  instrument.  It  is  probably  wiser  to 
abolish  final  examinations  than  to  follow  the  practise  of  excusing  students 
upon  the  basis  of  high  daily  grades  and  satisfactory  conduct.  In  con- 
sidering the  defense  of  the  written  examination  it  is  necessary  to  bear  in 
mind  that  the  existence  of  such  abuses  of  examinations  should  not  be  in- 
terpreted as  criticisms  of  examinations,  but  rather  of  the  teachers  and 
other  school  officials  who  use  them. 

7.  Examinations  as  objectives.  Examinations  have  been  criticised 
because  they  tend  to  become  immediate  educational  objectives.  This  is 
not  wholly  undesirable.  If  the  examination  consists  of  appropriate 
questions  a  pupil  who  studies  to  pass  the  examination  will  be  directing 
his  energies  to  appropriate  ends.  It  is  only  when  the  examinations  tend 
to  become  exclusive  objectives  or  consist  of  inappropriate  questions  that 
this  criticism  has  much  force.  Furthermore,  it  should  be  recognized  that 
this  criticism  also  is  not  a  criticism  of  examinations  as  such,  but  rather  of 
the  use  that  is  made  of  them  by  teachers  and  other  school  officials. 

8.  Effect  of  examinations  upon  health  of  students.  While  it  is 
undoubtedly  true  that  the  preparation  for  examinations  and  the  writing 
of  them  frequently  make  enormous  drains  on  the  energies  of  students, 
we  have  no  carefully  conducted  investigation  of  the  actual  effect  upon  their 
health.  To  one  who  observes  the  enormous  expenditures  of  time  and 
energy  which  students  devote  to  social  and  athletic  activities,  it  is  difficult 
to  believe  that  examinations  are  in  general  more  injurious  to  the  health  of 
students  than  many  other  activities  in  which  they  are  permitted  and  even 
encouraged  to  engage.  It  is  undoubtedly  true  that  in  the  case  of  some 
individual  students  examinations  make  a  heavier  drain  upon  their  energies 
than  they  should  be  asked  to  bear.  However,  here  again  it  should  be  real- 
ized that  this  criticism  is  not  fundamentally  the  criticism  of  examinations 
but  rather  of  setting  very  long  examinations  or  of  placing  extreme  emphasis 
upon  them  by  making  the  final  "grade"  depend  wholly  or  very  largely 
upon  the  examination  "grade." 
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9.  Time  devoted  to  examinations  a  profitable  investment.  After  a 
teacher  has  instructed  his  students  to  the  best  of  his  ability  some  will  have 
failed  to  learn;  some  will  have  learned  certain  details  but  neglected  others; 
and  a  few  may  have  mastered  all  of  the  assignment.  It  becomes  necessary 
at  some  time  for  each  teacher  to  check  up  the  work  of  his  students,  in 
order  to  know  which  students  need  supplementary  or  remedial  instruction. 
Only  by  thus  ascertaining  the  extent  to  which  his  students  have  achieved 
can  the  teacher  make  his  instruction  fit  the  needs  of  his -class.  For  these 
reasons  the  time  which  is  devoted  to  the  construction  of  an  examination, 
giving  it,  and  rating  the  papers  should  be  considered  a  legitimate  expendi- 
ture of  the  teacher's  time;  in  fact,  in  some  cases  it  is  doubtless  among  the 
most  profitable  of  the  time  expenditures  which  the  instructor  makes.4 
In  fact,  the  instructor  may  very  profitably  do  more  than  simply  rate  the 
examination  papers.  In  the  case  of  students  receiving  low  marks,  their 
answers  may  be  studied  in  order  to  ascertain  in  what  ways  and  why  they 
have  failed.  Such  information  will  frequently  be  exceedingly  illuminating 
to  the  instructor.  This  argument  applies  more  forcibly  to  examinations 
or  "tests"  given  at  intervals  throughout  the  course  than  to  final  examina- 
tions, but  even  in  the  latter  case  the  instructor  will  frequently  receive 
many  helpful  suggestions  for  improving  his  teaching. 


4This  argument  applies  only  to  a  very  limited  extent  to  examinations  given  at  the  end 
of  a  course. 
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CHAPTER  III 
METHODS  OF  IMPROVING  EXAMINATIONS 

1.     Increasing  objectivity  in  marking  of  examination  papers.     The 

principal  source  of  error  in  the  'grades"  assigned  to  examination  papers 
is  the  subjectivity  of  the  marking.  This  fault  has  been  sufficiently  demon- 
strated. It  is,  however,  possible  to  increase  materially  the  objectivity  of 
written  examinations.  This  may  be  accomplished  in  two  ways:  (1) 
improving  the  examination  questions,  and  (2)  introducing  improved  meth- 
ods of  marking  examination  papers. 

A.     Increasing  the  accuracy  of  "grades"  by  improving  examination 
questions. 

(1.)  General  Methods.  The  questions  should  be  so  stated  that  all 
pupils  will  interpret  them  in  the  same  way.  When  a  question  is  am- 
biguous the  performances  which  different  pupils  give  are  not  comparable. 
Pupils  who  are  so  unfortunate  as  to  interpret  a  question  in  a  way  not  in- 
tended by  the  instructor,  can  receive  no  credit  for  their  answers  unless 
the  instructor  recognizes  the  various  interpretations.  Even  in  this  case 
all  answers  are  not  comparable  because  some  are  judged  with  reference  to 
one  interpretation  and  others  with  reference  to  another.  A  striking 
illustration  of  the  difficulty  of  eliminating  all  ambiguous  questions  occurs 
in  Form  7  of  the  Army  Alpha  Intelligence  Test.  The  exercise  asks  the 
person  taking  the  test  to  tell  whether  "cleave"  and  "split"  mean  the  same 
or  opposite.  Both  answers  are  correct  because  "cleave"  has  two  meanings 
which  are  exactly  opposite.  Apparently  this  ambiguity  was  not  detected 
although  the  test  was  prepared  with  care  by  well  trained  persons.  How- 
ever, in  most  cases,  teachers  will  be  able  to  avoid  ambiguous  exercises  if 
they  are  subjected  to  a  careful  scrutiny.  All  questions  should  be  so  stated 
that  they  will  be  clearly  understood. 

Pupils  should  be  given  definite  instructions  concerning  the  methods 
of  work  to  be  employed.  They  should  be  told  whether  they  are  to  work 
rapidly  or  slowly.     In  some  cases  it  is  advisable  for  the  examiner  to  say 

at  appropriate  intervals,  "It  is  now  time   to   begin   on   question  ." 

Perhaps  students  should  not  be  forced  to  comply  with  these  directions, 
but  they  will  tend  to  prevent  an  improper  distribution  of  time  over  the 
list  of  questions.  In  questions  in  which  pupils  are  asked  to  "discuss"  or 
"explain"  the  completeness  of  the  discussion  desired  should  be  indicated. 
In  arithmetic  pupils  should  be  given  directions  concerning  the  form  in 
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which  their  work  is  to  appear.  If  it  is  to  be  copied  in  a  particular  form  all 
should  do  this;  if  it  may  be  left  in  the  form  used  to  calculate  the  answer  all 
pupils  should  leave  it  in  this  form. 

From  the  standpoint  of  securing  accurate  measurements  it  is  best  to 
set  an  examination  which  is  relatively  hard.  If  more  than  10  percent  of 
the  class  make  perfect  "grades"  the  examination  is  "too  easy,"  i.e.,  it  is 
not  difficult  enough  to  yield  accurate  measures  of  the  achievements  of 
bright  pupils.  Unless  the  class  is  a  selected  group  it  is  not  possible  for 
more  than  10  per  cent  of  the  best  pupils  to  possess  exactly  the  same  degree 
of  achievement.  The  examination  should  be  long  enough  so  that  all  pupils 
will  be  employed  for  practically  the  entire  period.  The  examination  is 
not  only  useful  as  a  device  for  measuring  the  improvements  of  pupils,  but 
also  valuable  as  a  school  exercise.  No  pupil  should  be  permitted  to  waste 
any  of  the  time  set  aside  for  the  examination. 

(2.)  New  examinations:  use  of  questions  permitting  only  one  correct 
answer.  The  marking  of  examination  papers  becomes  subjective  when 
the  scorer  is  asked  to  exercise  judgment  in  determining  the  credit  to  be 
given  for  the  pupil's  performance.  In  spelling,  a  pupil's  performance  is 
either  right  or  wrong  and  our  practise  is  to  allow  no  credit  for  a  per- 
formance which  is  not  entirely  correct.  Thus,  the  marking  of  an  examina- 
tion paper  in  spelling  is  highly  objective  because  the  scorer  is  not  asked 
to  exercise  judgment  concerning  the  quality  of  the  performance.  A  high 
degree  of  objectivity  may  be  attained  also  in  the  operations  of  arithmetic 
by  agreeing  to  give  no  credit  for  examples  partly  correct.  In  other  subject 
matter  fields  we  are  accustomed  to  ask  some  questions  which  call  for  specific 
facts  and,  hence,  admit  of  only  one  correct  answer.  It  has  been  claimed 
that  such  questions  appeal  only  to  the  pupil's  memory  and  that  they  do 
not  yield  an  index  of  his  acquaintance  with  principles  and  of  his  ability  to 
organize  and  apply  his  knowledge.  In  order  to  reach  this  phase  of  his 
education  we  have  asked  the  pupil  to  "discuss,"  "tell  why,"  "compare," 
etc.  When  a  pupil  is  asked  to  formulate  an  answer  consisting  of  one  or 
more  sentences  it  is  difficult  or  impossible  to  classify  the  performance  as 
either  right  or  wrong.  When  scorers  are  asked  to  exercise  judgment  in 
evaluating  such  performances,  wide  differences  of  opinion  exist.  In  order 
to  overcome  this  subjectivity  of  marking,  it  has  recently  been  proposed  that 
we  can  measure  a  pupil's  acquaintance  with  principles  and  ideas  by  means 
of  certain  types  of  exercises  which  permit  of  only  one  correct  answer. 
Such  exercises  have  been  used  in  our  standardized  educational  tests,  and 
it  is  now  suggested  that  they  be  used  by  teachers  in  their  examinations. 
Four  types  of  such  exercises  will  be  considered. 
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a.  True-false  exercises.  Instead  of  asking  the  pupil  to  formulate 
an  answer  in  response  to  a  question,  we  ask  him  to  tell  whether  a  given 
statement  is  true  or  false.  For  example,  instead  of  the  question,  "Why 
did  the  Puritans  come  to  America  in  the  17th  Century?"  we  ask  whether 
the  following  statement  is  true  or  false,  "The  Puritans  came  to  America 
in  the  17th  century  seeking  wealth."  The  pupils  may  give  his  answer  to 
this  true-false  exercise  by  writing  a  plus  sign  if  he  considers  it  true  and  a 
minus  sign  if  he  considers  it  false.  The  mental  processes  required  in  answer- 
ing such  exercises  do  not  appear  to  be  the  same  as  those  which  occur  in 
answering  questions  of  the  usual  type.  However,  experimental  evi- 
dence indicates  that  there  is  a  high  correlation  between  the  scores  which 
pupils  make  on  a  true-false  examination  and  their  acquaintance  with  ideas 
and  principles  as  determined  by  our  ordinary  examinations. 

Directions  for  constructing  a  true-false  examination.1  1.  In  con- 
structing true-false  exercises,  one  may  prepare  a  list  of  statements  which 
cover  in  some  detail  the  portion  of  the  subject  on  which  the  pupils  are  to  be 
examined.  After  such  a  list  has  been  prepared,  some  of  the  statements 
can  easily  be  changed  so  that  they  are  false.  The  untruth  of  a  statement 
should  not  be  too  obvious  or  it  will  be  worthless  for  testing.  Also  state- 
ments should  be  selected  which  require  an  acquaintance  with  the  subject 
in  order  to  determine  their  truth  or  falsity. 

2.  In  a  true-false  examination  some  of  the  statements  should  be  true 
and  some  false,  and  the  number  of  true  statements  should  approximate  the 
number  of  false  statements.  They  should  be  arranged  so  that  there  is  no 
regular  sequence  between  true  statements  and  false  statements. 

3.  Since  the  pupil  can  give  his  responses  very  quickly,  the  examina- 
tion should  consist  of  not  less  than  50  statements.  A  true-false  examina- 
tion of  100  statements  can  be  given  in  the  time  usually  devoted  to  an  ordi- 
nary examination. 

4.  The  examination  should  be  mimeographed  or  printed  so  that  each 
pupil  will  have  a  copy.  He  may  give  his  answers  in  the  margins  of  the 
sheets,  or,  if  it  is  desired  to  use  the  same  set  of  papers  with  another  group 
of  pupils,  he  may  be  given  a  sheet  of  paper  on  which  there  are  numbered 
blanks.  The  pupils  will  then  be  asked  to  record  in  the  blanks  their 
answers  to  the  corresponding  exercises.  A  less  desirable  plan,  which  may 
be  followed  when  it  is  not  possible  to  secure  mimeographed  copies  of  the 
examination,  is  to  read  the  statements  to  the  pupils  and  have  them  record 
their  answers  in  numbered  blanks.     The  disadvantage  of  this  plan  is  that 


xTwo  sample  true-false  examinations  are  given  in  the  appendix. 
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the  pupils  do  not  have  a  satisfactory  opportunity  to  study  the  statements. 
Also  the  class  may  give  some  indication  of  the  answer  if  a  statement 
appeals  to  them  as  being  ridiculous. 

5 .  The  pupils  should  be  given  specific  directions  in  regard  to  answering 
exercises  about  which  they  are  uncertain.  One  writer2  has  suggested 
that  the  pupils  be  instructed  to  guess  when  uncertain  concerning  the  truth 
or  falsity  of  the  statement.  Another  writer3  who  has  used  this  type  of 
examination  instructed  the  pupils  as  follows:  "First,  go  through  the  list 
quickly  and  mark  all  that  you  know  for  certain,  then  go  back  and  study 
out  the  harder  ones.  Do  not  guess;  the  chances  are  against  you  on  guess- 
ing. Don't  endanger  your  score  by  gambling  on  those  questions  about 
which  you  know  nothing."     This  second  procedure  is  probably  the  better. 

The  scoring  of  a  true-false  examination.  Since  only  two  responses 
are  possible,  it  is  obvious  that  a  pupil  may  give  a  correct  response  as  the 
result  of  chance.  In  order  to  take  this  possibility  into  account,  a  pupil's 
score  on  an  examination  of  this  type  is  the  number  of  exercises  answered 
correctly  minus  the  number  answered  incorrectly.  Exercises  not  at- 
tempted are  not  counted.  . 

b.  "Yes"  and  'no"  exercises.  These  exercises  are  just  what  their 
name  implies.  Each  is  answered  correctly  by  "yes"  or  "no."  No  other 
answers  are  appropriate.  The  administration  and  scoring  are  similar  to 
those  of  the  "true-false"  exercises,  of  which  they  may  be  considered  a 

special  type. 

In  constructing  "yes"  and  "no"  exercises  care  must  be  taken  to  avoid 
questions  for  which  either  of  the  two  possible  answers  is  absurd.  ^  In  such 
case,  obviously,  a  correct  answer  will  not  furnish  satisfactory  evidence  of 
the  pupil's  achievement  in  a  school  subject.  Approximately  ^ the  ^same 
number  of  exercises  should  call  for  the  answer  of  "yes"  as  for  "no,"  and 
they  should  be  arranged  in  no  regular  order  of  sequence. 

c.  Recognition  exercises.  Exercises  in  which  the  pupil  is  asked  to 
choose  from  a  number  of  proposed  answers  have  also  been  used  to  make 
the  scoring  objective.  This  type  of  question  has  been  called  the  "recog- 
nition exercise."  It  has  been  used  in  standardized  silent  reading  tests  and 
in  a  number  of  our  group  intelligence  tests,  and  may  be  illustrated  by  the 
following: 

"The  first  president  of  the  United  States  was:  Christopher  Colum- 
bus, Benjamin  Franklin,  George  Washington,  Thomas  Jefferson." 

2McCall,  W.  A.    "A  new  kind  of  school  examination,"  Journal  of  Educational  Research, 

'Woodmen*!).  "The  measurement  of  college  work,"  Educational  Administration  and 
Supervision,  7:  301-34,  September,  1921. 
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The  correct  answer  may  be  indicated  by  an  underline  or  by  some  other 
mark.  If  only  one  of  the  answers  proposed  may  be  considered  correct 
the  marking  of  such  exercises  will  be  highly  objective. 

Directions  for  constructing  a  recognition  examination.4  In  con- 
structing this  type  of  examination  none  of  the  proposed  answers 
should  be  too  obviously  incorrect.  An  exercise  can  yield  an  indication  of 
a  pupil's  achievement  only  when  he  is  forced  to  exercise  judgment  in  de- 
termining which  of  the  proposed  answers  is  suitable.  For  example,  the 
illustrative  exercise  given  above  would  be  practically  worthless  for  testing 
purposes  if  all  the  names,  except  that  of  George  Washington,  were  of  per- 
sons living  today  or  of  persons  who  had  had  no  connection  with  our  na- 
tional life.  In  applying  this  type  of  exercise  to  the  field  of  arithmetic  the 
proposed  answers  should  include  erroneous  answers  which  pupils  are  in- 
clined to  give.  For  example,  if  the  exercise  called  for  the  quotient  of  two 
fractions,  one  of  the  proposed  answers  should  be  the  product  of  the  frac- 
tions and  another  should  be  their  sum,  and  perhaps  another  should  be  the 
fraction  obtained  by  taking  the  sum  of  the  numerator  as  a  new  numerator 
and  the  sum  of  the  denominator  for  a  new  denominator.  When  the 
correct  answer  is  included  in  a  group  of  such  answers  as  these,  the  pupil 
who  does  not  know  how  to  find  the  quotient  of  such  fractions  will  be 
unable  to  determine  the  correct  answer  except  as  a  matter  of  chance.  On 
the  other  hand,  if  all  of  the  answers  except  the  correct  one  were  integers 
or  were  so  large  that  they  were  obviously  incorrect,  a  bright  pupil  who 
knew  nothing  about  division  of  fractions  would  be  able  to  select  the  cor- 
rect answer.  The  correct  answer  should  not  always  be  found  in  the  same 
position;  sometimes  it  should  be  first,  sometimes  last,  and  sometimes  in 
an  intermediate  position.  As  in  the  case  of  the  true-false  examination, 
a  recognition  examination  should  consist  of  a  large  number  of  exercises. 

Examinations  of  this  type  should  be  mimeographed  or  printed  and  a 
copy  given  to  each  pupil.  He  should  be  given  definite  instructions  con- 
cerning the  method  of  work  to  be  followed.  It  is  probably  best  to  instruct 
him  to  work  through  the  test  rapidly,  answering  those  exercises  about 
which  he  is  certain.  He  should  then  go  back  over  the  list  and  try  the  more 
difficult  ones.  Not  fewer  than  four  proposed  answers  should  be  given. 
When  this  is  done  the  chances  are  slight  that  a  pupil  will  give  the  correct 
answer  by  guessing.  The  pupils  may  be  instructed  to  guess  if  they  do  not 
know,  since  the  chance  of  success  by  guessing  is  slight.  The  pupil's  score 
on  an  examination  of  this  type  may  be  taken  as  the  number  of  exercises 
done  correctly. 

4A  sample  recognition  examination  is  given  in  the  appendix. 
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A  somewhat  unusual  but  interesting  type  of  recognition  exercise  is 
that  described  as  a  "matching  contest."  In  this  a  pupil  is  given  two  lists 
of  statements,  the  first  numbered  1,  2,  3,  4,  5,  etc.,  the  second  marked  A, 
B,  C,  D,  E,  etc.  In  the  second  list,  there  is  a  statement  which  corres- 
ponds in  meaning  to  a  statement  in  the  first  list  and  the  pupil  is  to  pair 
these  statements,  marking  by  the  number  of  the  first  list  the  letter  of  the 
corresponding  statement  of  the  second.  For  example,  in  the  exercises 
given  below:  by  the  date  marked  (5)  1898  we  place  the  letter  B  to  indicate 
the  event  for  which  that  date  is  significant.  It  is  difficult  to  construct  such 
examinations  so  that  they  will  require  reasoning  on  the  part  of  the  student. 
Their  most  important  use  is  in  the  elementary  school  for  rapid  drill  in 
certain  phases  of  some  subjects,  such  as  definitions  in  geography  and 
grammar,  etc.  The  following  exercises,  selected  from  the  Spokane  United 
States  History  Test,  illustrate  the  use  of  such  an  examination  in  linking  a 
certain  date  or  person  with  the  corresponding  event. 

Lincoln's  Emancipation  Proclamation. 
Spanish  American  War. 
Beginning  of  World  War. 
Declaration  of  Independence. 
United  States  entered  World  War. 
Election  of  Washington  as  President. 
War  with  Mexico  began. 
Invention  of  the  cotton  gin. 
Lee's  surrender  at  Appomattox. 
Beginning  of  Civil  War. 


Destroyed  Spanish  fleet  in  Manila  Bay. 

Invented  the  telephone. 

Leading  Confederate  General. 

Wrote  the  Declaration  of  Independence. 

Invented  the  steamboat. 

Commanded  allied  armies  in  the  World  War. 

W7as  President  during  the  Civil  War. 

Commanded  American  Forces  in  the  World 
War. 
I.      Was  Revolutionary  patriot,  author,  and  in- 
ventor. 
J.     America's  most  famous  inventor. 


1. 

1846 

A. 

2. 

1865 

B. 

3. 

1863 

C. 

4. 

1917 

D. 

5. 

1898 

E. 

6. 

1789. 

F. 

7. 

1792. 

G. 

8. 

1776. 

H. 

9. 

1861. 

I. 

0. 

1914. 

J- 

1. 

Foch 

A. 

2. 

Lincoln 

B. 

3. 

Fulton 

C. 

4. 

Dewey 

D. 

5. 

Pershing 

E. 

6. 

Bell 

F. 

7. 

Edison 

G. 

8. 

Jefferson 

H. 

9.     Lee 


10.     Franklin 


33 


d.  Completion  exercises.  Pupils  may  be  asked  to  fill  in  the  words 
omitted  from  statements.     The  following  illustrates  this  type  of  exercise: 

"Fill  in  the  blanks.     Revenue  for  paying  the  war  debts  of  the  states 

after   the   Revolutionary  War  was   provided   by  a and  by 

due  largely  to influence. 

The  slavery  question  in  such  states  as  should  be  carved  from  the 
Louisiana  Territory  was  temporarily  settled  in by  the " 

Directions  for  constructing  completion  exercises.5  A  completion 
exercise  should  be  constructed  so  that  no  suggestion  will  be  given  of  the 
correct  words  to  be  written  in  the  blanks.  Furthermore,  the  facts  to  be 
supplied  should  be  important.  The  best  plan  is  to  prepare  a  list  of  im- 
portant statements  and  principles  covering  the  portion  of  the  subject 
over  which  the  pupils  are  to  be  examined  and  then  from  these  statements 
to  strike  out  a  certain  significant  word  or  phrase.  In  every  case,  if  it  is 
possible,  the  words  omitted  should  be  such  that  only  one  answer  will  be 
correct.  Since  little  writing  is  required  of  the  pupils  they  may  be  asked 
to  fill  in  as  many  as  one  hundred  blanks. 

The  scoring  of  completion  exercises  is  not  as  highly  objective  as  in 
the  two  types  mentioned  above.  Pupils  will  tend  to  write  a  variety  of 
words  in  the  blanks.  Different  words  may  have  almost  the  same  meaning, 
and  frequently  the  scorer  will  be  compelled  to  determine  whether  the  mean- 
ing of  some  word  is  sufficiently  near  to  that  of  the  correct  answer  to  justify 
giving  the  pupil  credit  for  having  answered  the  exercises  correctly.  How- 
ever, by  a  careful  selection  of  statements  and  of  the  omitted  words,  this 
subjectivity  may  be  greatly  minimized.  For  example,  in  the  sentence, 
"The  first  Continental  Congress  was  held  in ,"  only  one  possi- 
ble word  can  be  correct.  In  using  completion  exercises  it  is  necessary  to 
provide  each  pupil  with  a  mimeographed  or  printed  copy  of  the  examina- 
tion.    The  pupil's  score  is  the  number  of  blanks  filled  in  correctly. 

e.  Other  advantages  of  the  "new  examination."  Examinations  in 
which  true-false  questions  and  the  other  types  of  exercises  described  on 
pages  29-34  are  used  have  been  called  "new  examinations."  Such  ex- 
aminations have  certain  advantages  in  addition  to  increasing  the  ob- 
jectivity of  the  marking  of  the  papers.  There  will  be  a  large  saving  of 
time  both  for  pupil  and  teacher.     The  pupil  is  called  upon  to  do  little  or  no 


6A  sample  completion  examination  is  given  in  the  Appendix. 
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writing  in  giving  his  answers,  and  he  is,  therefore,  able  to  respond  to  a  large 
number  of  exercises.  The  scorer  will  have  little  or  no  occasion  to  exercise 
judgment  as  he  will  need  only  to  note  the  brief  responses  given  by  the 
pupils.  Thus,  the  labor  of  scoring  will  be  greatly  reduced.  The  saving 
of  time  in  the  giving  and  scoring  will  more  than  offset  any  additional  time 
that  may  be  expended  in  the  construction  of  the  "new examination." 
Another  advantage  is  that  the  examination  can  be  made  more  compre- 
hensive. It  is  traditional  for  examinations  to  consist  of  ten  questions.  A 
few  are  limited  to  a  smaller  number,  and  only  occasionally  do  we  find  ex- 
aminations consisting  of  more  than  ten  questions.  The  pupils  cannot 
write  upon  a  large  number  of  questions  in  the  time  allowed.  Thus,  the 
scope  of  traditional  examinations  is  necessarily  narrow.  We  have  stated 
that  true-false  examinations  should  include  not  fewer  than  50  exercises. 
Examinations  consisting  of  completion  exercises  or  recognition  exercises 
should  have  a  corresponding  length.  Thus,  the  "new  examination"  may 
be  made  distinctly  larger  in  scope.  Children  are  apt,  also,  to  be  interested 
in  the  new  examinations  which  are  distinctly  different  from  the  usual  type 
of  school  exercise,  and  which  make  an  appeal  somewhat  in  the  nature  of  a 
game.  In  being  relieved  of  much  writing,  which  especially  in  the  case  of 
young  children,  amounts  in  itself  to  a  laborious  exercise,  they  are  probably 
less  fatigued,  and  are  able  to  devote  all  their  energies  to  the  process  of 
thinking. 

f.  Limitations  of  the  "new  examination."  It  does  not  appear  likely 
that  the  "new  examination,"  consisting  of  the  type  of  exercises  we  have 
described,  should  replace  entirely  the  traditional  type  of  examination. 
The  "new  examination"  cannot  be  used  in  mathematics,  except  to  a  limited 
extent.  It  cannot  be  used  at  all  in  English  composition.  The  following 
questions  taken  from  Hahn's  Scale  for  Measuring  the  Ability  of  Children 
in  History  appear  to  require  mental  processes  distinctly  different  from 
those  for  which  the  "new  examination"  calls. 

"State  points  of  similarity  between  the  position  of  the  United  States 
in  1812  and  her  position  in  1912. 

"Arrange  the  following  events  in  order  of  cause  and  effect:  Force 
Bill,  the  Carpetbaggers,  15th  Amendment,  Negro  Rule  in  some  of  the 
Southern  States,  Ku  Klux  Klan. 

"Name  the  presidents  of  the  United  States  since  1892." 

Furthermore,  it  is  likely  that  pupils  would  miss  valuable  experience 
and  training  if  they  were  not  asked  at  times  to  "compare,"  "explain," 
"discuss,"  "define,"  or  "tell  why."  This  is  also  true  of  questions  in  which 
they  are  asked  to  summarize  material  presented  on  a  topic  or  to  apply 
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certain  principles  that  have  been  presented.  Hence,  it  is  difficult  to  con- 
ceive of  the  "new  examination"  being  a  complete  substitute  for  the  tra- 
ditional examination.6 

B.  Rules  for  marking  examination  papers.  The  marking  of 
ordinary  examination  papers  may  be  made  more  objective  by  following 
a  few  simple  rules.  It  is  advisable  often  to  use  the  "sorting  method." 
According  to  this  procedure  the  papers  as  they  are  read  are  sorted  into 
piles,  the  best  ones  being  placed  in  the  first  pile,  the  next  best  in  the  second, 
etc.  Five  distributions  will,  in  most  cases,  prove  sufficient,  but  as  the 
reading  goes  on,  if  papers  are  found  which  do  not  appear  to  belong  in  any 
of  the  piles,  additional  ones  may  be  started.  After  all  of  the  papers  have 
been  distributed  they  should  be  re-read,  taking  one  pile  at  a  time,  and  the 
papers  in  it  compared  with  each  other.  If  these  papers  do  not  possess 
approximately  the  same  value  changes  in  the  sorting  may  be  made.  After 
the  examiner  has  satisfied  himself  with  reference  to  the  sorting,  "grades" 
may  be  assigned  to  the  papers  in  the  different  piles.  Instead  of  actually 
sorting  the  papers  they  may  be  marked  with  a  symbol  to  indicate  the 
distribution,  i.e.,  mark  the  best  papers  5,  the  next  in  quality  4,  and  so  on. 
However,  when  this  is  done,  the  papers  on  which  the  same  symbol  has  been 
placed  should  be  brought  together  and  re-read. 

When  a  large  number  of  papers  are  to  be  rated  it  is  well  to  consider 
one  question  at  a  time.  In  doing  this  it  is  desirable  to  follow  essentially 
the  sorting  method,  or,  at  any  rate,  to  assign  marks  to  the  answers  as  read. 
All  of  the  papers  should  be  read  for  the  first  question  and  a  mark  assigned 
it.  Five  seems  a  good  number  for  approximately  perfect  answers,  3  for 
average,  and  1  for  very  poor  answers.  The  marks  of  2  and  4  and  even 
fractional  numbers  may  be  used  as  appropriate  descriptions  of  quality. 
After  the  marks  on  this  first  question  have  been  assigned,  those  papers 
which  have  been  given  the  same  marks  may  be  brought  together  and  re- 
read. The  same  procedure  should  be  followed  in  reading  each  of  the  other 
questions.  It  is  well  to  shuffle  the  papers  in  proceeding  from  one  question 
to  another  so  that  the  reader  will  not  be  influenced  by  previous  marks. 
When  the  questions  have  all  been  read,  the  marks  assigned  to  each  paper 
may  be  totaled,  and  on  this  basis  a  final  distribution  of  the  papers  made. 
Although  this  method  may  seem  to  require  an  almost  prohibitive  amount  of 
time  on  the  part  of  the  scorer,  the  increased  objectivity  of  the  "grades" 


6In  the  appendix  questions  selected  from  actual  examinations  given  in  Illinois  schools  are 
listed.  These  questions  in  their  demand  for  explanation,  discussion,  organization  of  material, 
exercise  of  judgment,  etc.,  show  that  these  traditional  examinations  call  for  a  distinct  type 
of  ability  not  required  by  any  form  of  the  "new  examination." 
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assigned  may  justify  its  adoption.  Especially  is  it  advised  in  those  cases 
where  emphasis  needs  to  be  placed  on  the  accuracy  of  examination  grades. 

The  marking  of  examination  papers  within  a  school  system  may  be 
made  more  objective  by  formulating  rules  with  reference  to  the  credit  to 
be  given  for  certain  types  of  answers.  For  example,  uniform  rules  may  be 
agreed  upon  with  reference  to  the  credit  given  for  correct  principle  when  the 
answer  is  wrong,  the  credit  given  for  answers  partially  right,  the  penalty, 
if  any,  for  misspellings,  grammatical  errors,  and  poor  handwriting.  In 
case  the  same  examination  is  given  by  two  or  more  teachers  some  specific 
rules  may  be  formulated  for  the  particular  examination.  If  time  permits, 
the  objectivity  of  the  marks  may  be  increased  by  having  the  papers  rated 
independently  by  two  or  more  teachers  and  using  the  average  of  the 
marks  assigned.  A  modification  of  this  procedure  is  to  have  one  teacher 
rate  all  of  the  answers  to  certain  questions  and  another  teacher  the  answers 
to  other  questions. 

F.  J.  Kelly7  describes  an  experiment  which  is  indicative  of  the  in- 
crease in  the  objectivity  of  the  marking  of  examination  papers  when  uni- 
form rules  are  adopted.  Six  fifth-grade  teachers  gave  the  same  examina- 
tion in  arithmetic  to  their  pupils.  Each  teacher  marked  the  papers  for 
her  own  pupils  but  did  not  record  the  marks  on  the  papers.  The  super- 
intendent then  asked  a  teacher,  who  was  unusually  systematic  in  marking 
examination  papers,  to  prepare  a  set  of  rules  to  be  followed  in  the  marking 
of  these  papers.  After  she  had  done  so,  she  marked  all  of  the  papers  in 
accordance  with  this  plan.  Then  the  teachers  who  had  first  marked  the 
papers  marked  them  a  second  time  following  her  plan.  This  provided  two 
marks  for  each  paper  given  by  the  classroom  teacher,  the  first  without 
following  any  systematic  plan  and  the  second  given  in  accordance  with 
the  rules  formulated.  Each  of  these  marks  was  compared  with  the  mark 
given  by  the  one  teacher  who  marked  all  of  the  papers.  In  Table  I,  the 
six  teachers  are  designated  by  the  letters  A,  B,  C,  D,  E,  and  F.  The  table 
is  read  as  follows:  When  no  rules  were  followed  teacher  A  marked  one 
paper  16  to  20  points  lower  than  the  "judge,"  one  paper  7  points  lower, 
two  papers  4  points  lower,  two  papers  2  points  lower,  agreed  with  the 
"judge"  on  one  paper,  etc.  The  differences  between  the  marks  given  when 
the  class-room  teachers  followed  no  rules  and  when  they  followed  the  rules 
formulated  are  very  striking.  In  the  first  instance  the  marks  assigned  by 
the  teacher  agreed  with  those  assigned  by  the  "judge"  in  only  5.5  percent 
of  the  cases,  while  in  the  second  instance  they  agreed  in  63.5  percent  of  the 


7Kelly,  F.  J.     "Teachers'   marks,"   Teachers   College,  Columbia  University,  Contri- 
butions to  Education,  No.  66,  p.  83,  1914. 
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Table   I.     Distributions   of    Differences    beween    Two   Sets   of 

Teachers'    Marks    on    Fifth-Grade    Arithmetic    Papers — 

First,  without  any  Effort  to  Unify  the  Methods 

Used,  and  Second,  by  a  Common  Standard 

(after  Kelly) 

Range  of 

Without  Standard 

With  Standard 

Differences 

A 

B 

C 

D 

E 

F 

Total 

A 

B 

C 

D 

E 

F 

Total 

21  or  more 

2 

1 

.... 

2 
1 

2 

'Y 

i 

i 

2 
2 
2 
1 
4 

1 

1 

1 

.... 

.... 

2 
3 
2 
1 
3 
2 
4 

4 

5 

5 

4 

7 

10 

11 

8 

18 

12 

14 

16 

13 

17 

10 

7 

9 

6 

4 

3 

2 

3 

3 

1 

2 

2 

5 

16  to  20 

1 

15 

14 

13 

1 

1 
1 

12 

1 

11 

1 

1 

1 
1 

10 

1 

9 

1 

2 
3 
1 
1 
1 
1 
2 
1 
2 

1 

2 
3 
2 

5 

1 
2 
1 

8 

1 

7 

1 

1 

2 
1 
2 
4 
2 
5 

4 

5 

1 

.... 

3 
1 

1 
2 

1 

1 

6 

5 

2 
2 
2 
1 
4 

4 

2 
3 
2 
4 
2 
.... 

1 

1 

1 
1 
1 
1 
3 

1 

2 

2 

.... 

2 
3 
6 

2 
2 

4 

2 

1 

1 
.... 

1 

26 

3 

2 

3 

17 
16 

139 

13 

5 
6 
8 
4 
2 

3 

1 
1 
4 

16 

2 
3 
2 
3 
1 
1 

1 

3 

5 

16 

2 

1 
7 
1 

29 

1 

2 

1 

2 

4 
2 

22 

5 

1 

1 

3 

30 

.... 

2 
2 
1 

0 

1 

2 

1 

2 
6 
9 

5 
2 

1 

3 

4 

1 

3 
2 
1 

1 

5 

6 

7. . 

8.. 

9 

1 
1 

10.. 

1 

11 

1 

1 

12.. 

1 

1 

.... 

1 

1 
1 

13.. 

1 

14 

15 

1 

1 

1 

1 

16  to  20.. 

? 

1 

3 

1 

Totals 

Medians 

35 
+3 

41 
0 

35 
+  1 

36 
+6 

39 
-1 

33 
-4 

219 

+  1 

35 

41 

35 

36 

39 

33 

219 

38 


cases.     This  indicates  a  very  marked  increase  in  the  objectivity  of  the 
marking  of  the  papers. 

2.  Increasing  the  objectivity  of  the  norms  for  translating  examination 
scores  into  school  marks.  The  use  of  examination  scores,  when  expressed 
in  terms  of  percents,  as  school  marks,  introduces  errors  which  are  as  serious 
as  the  errors  arising  from  the  subjectivity  of  the  marking  of  examination 
papers.  (See  page  9).  For  standardized  educational  tests  we  have 
objective  norms  with  which  to  compare  the  scores.  Since  a  single  ex- 
amination is  given  to  only  relatively  few  pupils,  it  is  not  possible  to  secure 
objective  norms  by  the  procedure  followed  in  standardizing  educational 
tests.  It  is,  however,  possible  to  reduce  greatly  the  errors  in  school 
marks  arising  from  this  source.  The  greatest  assistance  in  this  connection 
will  be  secured  by  recognizing  explicitly  that  the  scores  assigned  to  ex- 
amination papers  and  school  marks  are  different.  Occasionally  they  may 
coincide  numerically,  but  they  are  fundamentally  different.  When  this 
distinction  is  recognized  it  will  be  possible  for  a  teacher  to  introduce  ob- 
jective elements  into  the  norms  that  are  used  as  a  basis  for  translating 
examination  scores  into  school  marks. 

A  teacher  should  first  determine  whether  or  not  his  class  is  typical. 
The  giving  of  a  general  intelligence  test  will  be  helpful  in  this  connection. 
A  distribution  of  their  I.  Q.'s  may  be  considered  a  very  reliable  index  of  the 
composition  of  the  group.  If  the  median  I.  Q.  of  a  class  is  below  100  the 
teacher  may  know  that  he  has  poor  pupil  material.  If  the  median  I.  Q. 
is  above  100  he  may  know  that  the  class  consists  of  better  pupils  than  the 
average.  If  there  is  a  relatively  large  number  of  low  I.  Q's  it  may  be  ex- 
pected that  there  will  be  an  unusually  large  number  of  low  "grades." 
Thus,  by  means  of  the  intelligence  quotient  and  in  other  ways,  the  teacher 
may  come  to  know  the  general  status  of  his  class. 

The  method  to  be  followed  in  translating  the  examination  scores  into 
school  marks  may  be  illustrated  as  follows:  Suppose  that  a  set  of  ex- 
amination papers  has  been  rated  in  terms  of  percents.  The  score  placed 
upon  a  paper  simply  describes  the  percent  of  the  examination  which,  in 
the  judgment  of  the  scorer,  the  pupil  has  answered  correctly.  The  scores 
should  then  be  assembled  in  some  such  way  as  that  indicated  below: 
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56 

69 

47 

55 

69 

35 

44 

55 

68 

75 

27 

34 

42 

52 

63 

74 

25 

32 

40 

50 

60 

70 

39 


There  are  23  pupils  in  this  class.  The  scores  have  been  arranged  in  six 
groups.  It  would  be  better  to  make  five  groups  if  the  school  uses  a  system 
of  five  marks.  The  median  point  score  is  55.  If  the  class  is  an  average 
one  this  median  score  of  55  should  be  translated  into  the  median  or  average 
"grade''  which  the  school  recognizes.  If  the  "grades"  are  reported  in 
terms  of  percents  and  the  passing  mark  is  75  the  average  grade  will  usually 
be  approximately  85.  If  the  class  is  known  to  possess  superior  ability 
the  median  score  of  55  should  be  translated  into  a  higher  "grade."  On  the 
other  hand,  if  the  class  is  known  to  be  decidedly  below  average  in  ability, 
55  should  be  translated  into  a  lower  grade,  perhaps  as  low  as  77  or  78. 
In  an  extreme  case  it  might  be  translated  even  into  the  passing  grade  of 
75.  The  translation  of  the  median  score  into  a  grade  furnishes  a  basis  for 
translating  the  other  scores.  In  general,  the  scores  at  the  lower  end  of  the 
distribution  will  be  translated  into  "grades"  below  passing.  The  percent 
of  pupils  who  receive  such  "grades"  will  vary  widely  with  different  classes. 

There  is  a  somewhat  prevalent  opinion  that  the  normal  probability 
curve  fixes  the  percent  of  pupils  who  should  receive  "grades"  below  the 
passing  mark.  This  is  a  mistaken  notion.  The  normal  probability 
curve  tells  us  nothing  concerning  the  percent  of  pupils  who  should  receive 
any  "grade."  It  is  true  that  from  a  statistical  point  of  view  there  are 
certain  divisions  of  the  curve  which  are  convenient.  If  the  base  line  of 
the  curve  is  limited  to  a  length  equal  to  five  times  the  standard  deviation 
(5a)  and  this  distance  is  divided  into  intervals  of  l<r  and  perpendiculars 
are  erected  at  the  division  points,  the  percent  of  cases  falling  in  each 
division  of  the  curve  will  be  7,  24,  38,  24,  and  7.  It  has  been  suggested 
that  these  numbers  define  the  percent  of  pupils  who  should  receive  grades 
of  A,  B,  C,  D,  and  E,  respectively.  Similar  proposals  have  been  made 
for  other  plans  of  dividing  the  normal  probability  curve. 

No  plan  of  dividing  the  normal  probability  curve  can  claim  to  be 
distinctly  superior  to  any  other  plan.  One  may  accept,  without  being 
inconsistent,  the  assumption  that  the  "grades"  for  a  large  group  of  pupils 
should  conform  to  the  normal  probability  curve,  and  at  the  same  time 
refuse  to  accept  any  particular  proposed  specifications  as  to  the  percent 
of  pupils  who  should  receive  each  "grade."  For  example,  a  distribution 
of  50  percent  A's,  25  percent  B's,  10  percent  C's,  9  percent  D's,  and  6 
percent  F's  is  not  necessarily  inconsistent  with  the  assumption  that 
accurate  measures  of  achievement  tend  to  form  a  normal  distribution. 
However,  in  this  case,  it  would  be  necessary  to  define  a  "grade"  of  A  as 
being  a  "grade"  which  means  that  the  pupil  is  above  average  ability 
or,  in  other  words,  that  all  pupils  who  are  average  or  above  are  given  a 
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"grade"  of  A  without  any  attempt  to  distinguish  between  their  achieve- 
ment. Thus  a  "grade"  of  A  would  represent  a  wide  range  of  achieve- 
ment. On  the  other  hand,  "grades"  of  B,  C,  D,  and  F  represent  narrow 
ranges  of  achievement.  Thus,  the  different  "grades"  would  not  repre- 
sent equal  ranges  of  achievement.  This,  however,  for  certain  pur- 
poses, may  not  be  entirely  undesirable. 

The  range  of  achievement  which  a  "grade"  shall  represent,  or,  in 
other  words,  the  percents  of  pupils  who  in  the  long  run  shall  receive 
the  different  "grades,"  is  a  matter  of  school  policy.  This  should  be 
determined  by  the  school.  Perhaps  the  best  way  to  define  the  range  of 
achievement  which  a  "grade"  is  to  represent  is  in  terms  of  the  percent 
of  pupils  who,  in  the  long  run,  will  receive  the  "grade."  Undoubtedly, 
at  the  present  time,  there  is  considerable  variation  from  system  to  sys- 
tem in  respect  to  the  range  of  achievement  that  is  represented  by  the 
different  "grades."  If  a  system  can  reach  a  definite  agreement  con- 
cerning the  percent  of  pupils  who  should  receive  each  mark  a  step  will 
be  taken  in  the  direction  of  making  objective  the  norms  which  are  used 
in  translating  examination  scores  into  school  marks.  If  only  five  marks 
are  used  a  school  would  probably  not  be  far  from  the  general  practise 
if  the  percents  of  pupils  receiving  these  "grades"  were  defined  as  7,  24, 
38,  24,  and  7.  However,  in  establishing  this  definition  it  should  be 
distinctly  recognized  that  it  is  done  as  a  matter  of  school  policy  instead  of 
being  forced  upon  the  school  by  the  nature  of  the  normal  probability  curve. 

3.  Securing  closer  agreement  of  examinations  with  educational 
objectives.  One  of  the  weaknesses  of  examinations  set  by  teachers  and 
other  school  officials  is  that  questions  are  frequently  asked  which  call  for 
non-important  facts  or  which  are  not  in  agreement  with  the  emphasis 
that  has  been  placed  upon  the  course  during  the  term.  By  creating  a 
proper  attitude  toward  examinations  on  the  part  of  teachers  much  can  be 
accomplished  toward  bringing  the  examinations  into  agreement  with  our 
educational  objectives.  One  who  sets  an  examination  should  try  to  have 
it  include  the  more  important  points  that  have  been  taken  up  during  the 
term,  and  should  make  a  distinct  effort  to  omit  all  points  that  are  unim- 
portant. In  the  case  of  a  few  school  subjects  we  now  have  available  a 
partial  list  of  minimum  essentials.  One  setting  an  examination  may  use 
such  lists  in  determining  the  relative  importance  of  different  topics.  Text- 
books, also,  in  that  they  represent  the  judgment  of  experts  concerning 
what  topics  are  most  important,  are  valuable  guides  in  this  connection. 
Those  topics  which  are  treated  briefly  or  are  obviously  assigned  to  a  place 
of  minor  importance  should  not  be  included  in  an  examination. 
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CHAPTER  IV 
SUMMARY 

1.  Probably  the  most  prominent  function  of  the  written  examina- 
tion is  that  it  provides  an  opportunity  for  learning,  both  in  the  preparation 
for  the  examination  and  in  the  actual  writing  of  it.  The  pupil  who  is 
not  required  to  take  examinations  is  missing  an  important  part  of  his 
educational  opportunities.  The  importance  of  this  function,  however, 
varies  with  the  type  of  examination.  Examinations  which  call  merely 
for  facts  or  in  which  the  pupil  is  not  required  to  formulate  an  answer 
consisting  of  several  sentences  offer  only  a  limited  educational  oppor- 
tunity. Incidentally  it  may  be  noted  that  this  is  one  weakness  of  the  "new 
examination." 

2.  A  second  function  of  written  examinations  is  the  measurement 
of  the  achievements  of  pupils.  As  we  have  pointed  out,  measurements 
are  essential  to  the  organization  and  administration  of  schools  as  well  as 
to  class-room  instruction,  and  examinations  do  secure  a  type  of  information 
relative  to  the  achievements  of  pupils  which  can  be  secured  in  no  other 
way.  Although  written  examinations  possess  these  functions  it  should 
be  remembered  that  it  is  only  when  examinations  are  properly  used  and 
not  abused  that  the  fulfillment  of  these  functions  may  be  expected. 

3.  The  written  examination  furnishes  an  effective  motive.  It  is 
true  that  some  criticisms  may  be  offered  against  the  use  of  the  exami- 
nation to  motivate  the  work  of  the  school,  but  this  use,  unless  carried  to  an 
extreme,  is  probably  not  harmful. 

4.  Final  examinations  have  a  positive  value  in  that  they  call  for 
intensive  review  and  organization  of  the  content  of  the  course  on  the  part 
of  the  student.     This  emphasizes  an  important  phase  of  learning. 

5.  In  certain  school  subjects  standardized  objective  tests  are  su- 
perior to  written  examinations  set  by  the  teacher  as  instruments  for  meas- 
uring the  achievements  of  pupils.  Under  certain  conditions  teachers' 
estimates  and  daily  grades  may  be  more  accurate  measures  of  achieve- 
ments than  can  be  secured  by  means  of  written  examinations.  However, 
in  view  of  the  other  functions  of  written  examinations,  we  are  justified 
in  asserting  that  in  general  there  is  no  satisfactory  substitute  for  them. 
The  written  examination  is  more  important  in  some  school  subjects  than 
in  others,  and  frequently  it  furnishes  the  teacher  with  a  type  of  measure- 
ment of  the  achievements  of  pupils  that  cannot  be  secured  by  any  other 
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means.     In  addition,  it  is  necessary  to  recognize  that  certain  types  of 
written  examinations  fulfill  other  important  functions. 

6.  In  arriving  at  final  "grades,"  teachers'  estimates,  daily  "grades," 
and  "grades"  earned  on  short  quizzes  should  be  combined  with  examina- 
tion "grades."  No  general  rule  can  be  stated  concerning  the  weight  that 
should  be  given  to  each  of  these  types  of  measure  in  arriving  at  final 
"grades."  Under  certain  conditions  and  in  some  school  subjects  the  final 
examination  should  receive  considerable  weight,  perhaps  as  much  as  50  or 
60  percent.     In  other  cases  it  should  be  given  a  much  smaller  weight. 

7.  Teachers  should  give  more  attention  to  the  formulation  of  ex- 
amination questions.  Relatively  unimportant  topics  or  those  which  have 
not  received  emphasis  during  the  term  should  be  omitted.  Ambiguous 
statements  should  be  avoided.  The  questions  also  should  be  varied  so 
that  they  demand  different  types  of  mental  ability,  memory,  reasoning, 
organization,  etc.  Pupils  should  be  given  directions  concerning  methods 
of  work. 

8.  Examinations  should  be  long  enough  so  that  practically  no  pupils 
will  finish.     In  this  way  a  measure  of  rate  of  work  will  also  be  obtained. 

9.  The  examination  should  be  considered  a  part  of  the  regular  school 
work.  A  written  examination  has  a  unique  educational  value.  Pupils 
who  are  excused  from  it  or  who  are  not  kept  occupied  throughout  the  entire 
period  are  being  deprived  of  an  important  educational  opportunity. 

10.  A  systematic  procedure  should  be  followed  in  marking  the 
examination  papers.  Explicit  rules  should  be  formulated  relative  to  the 
effect  of  misspelling,  poor  English,  poor  handwriting,  and  so  forth,  upon 
the  "grade"  assigned  to  a  paper.  The  rule  should  cover  also  credit  for 
correct  principle  and  partial  credit  for  answers  partially  right.  The 
"sorting  method"  of  marking  is  recommended. 

11.  The  difference  between  scores  and  "grades"  should  be  recog- 
nized. Objective  factors  should  be  introduced  into  the  norms  used  for 
translating  the  scores  into  "grades." 

12.  The  "new  examination"  has  many  advantages,  among  which  are 
its  increased  objectivity  of  marking  and  the  economy  of  time  for  both 
pupils  and  teachers.  The  "new  examination"  will  undoubtedly  prove 
useful,  but  its  limitations  must  not  be  overlooked. 

13.  Traditional  examinations  call  for  the  functioning  of  distinct  types 
of  mental  ability  not  demanded  to  so  great  a  degree  in  any  other  kind  of 
school  work,  and  should  not,  especially  in  the  case  of  final  examinations, 
be  abolished,  nor  replaced  entirely  by  any  other  form  of  school  exercise. 
Teachers  and  pupils,  more  and  more,  should  be  impressed  with  their 
unique  educational  value. 
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APPENDIX 

(The  following  "new  examinations"  are  given  for  purposes  of  illustration.      They  may  in- 
clude several  exercises  which  will  prove  unsuitable  when  given  to  pupils.) 

TRUE-FALSE  EXAMINATION  IN  PHYSIOLOGY 

Prepared  by 

Bureau  of  Educational  Research 

University  of  Illinois 


Name Boy  or  Girl 

Age  last  birthday Next  birthday  will  be 

Grade Date City State. 

School Teacher 


.19. 


Below  you  will  find  a  number  of  statements.  Some  of  these  statements  are  true,  others 
are  not  true.  Read  each  statement  carefully,  then  if  it  is  true  mark  a  plus  (+)  in  the  column 
to  the  right  of  the  sentence.  If  the  statement  is  not  true  mark  a  minus  (  — )  in  the  column 
to  the  right. 


EXAMPLES 

Read  the  statement  below  very  carefully. 

1.  Fats  will  form  a  lasting  mixture  with  water. 

This  is  not  a  true  statement  so  you  will  place  a  minus  (  — )  sign  in  the  column 
Now  read  the  second  sentence. 

2.  The  layer  of  fat  just  beneath  the  skin  is  more  than  one-tenth  of  an 
inch  thick. 

This  is  a  true  statement  so  you  will  mark  a  plus  (+)  sign  in  the  column 

Now  read  the  third  sentence. 


3.     The  union  of  oxygen  with  any  substance  produces  heat. 

This  is  a  true  statement  so  you  will  mark  a  plus  (+)  sign  in  the  column. 
Now  read  the  fourth  sentence. 


4.    Nitrogen  constitutes  only  one-fifth  of  the  volume  of  the  air. 

This  is  a  false  statement  so  you  will  mark  a  minus  (  — )  sign  in  the  column, 


Answers  to  be 
written  here 
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PHYSIOLOGY 

1.  The  kidneys  vary  12  inches  to  16  inches  in  length. 

2.  The  external  poisoning  of  the  skin  by  poison  ivy  or  sumac  never  results 
seriously. 

3.  A  person  having  a  good  mind  must  necessarily  have  a  large  brain. 

4.  Color  blindness  is  more  prominent  in  men  than  women. 

5.  Plenty  of  fluids  should  be  drunk  at  the  time  of  eating  solid  food. 

6.  Bones  are  composed  of  animal  and  mineral  matter. 

7.  The  nails  are  hardened  outer  skin  or  epidermis. 

8.  The  use  of  alcohol  increases  the  tendency  to  commit  crime. 

9.  A  full  grown  person  contains  about  six  quarts  of  blood. 

10.  The  brain  is  almost  perfectly  spherical  in  shape. 

11.  The  kidneys  are  almost  perfectly  round. 

12.  All  animals  are  made  up  of  cells. 

13.  Substances,  like  glass,  which  permit  rays  of  light  to  pass  through  them 
readily  are  said  to  be  opaque. 

14.  The  sense  organs  of  smell  are  located  in  the  lining  of  the  cavity  of  the 
nose. 

15.  The  skin  is  composed  of  two  layers  of  tissue. 

16.  The  end  organs  for  taste  occur  in  the  mucuous  membrane  of  the  tongue. 

17.  To  extinguish  the  burning  clothing  of  a  person,  it  is  necessary  to  wrap 
him  in  something  to  exclude  the  air. 

18.  All  of  the  interior  of  the  spinal  cord  is  filled  with  gray  matter  containing 
nerve  cells. 

19.  The  great  difference  in  the  complexion  of  persons  is  due  largely  to  the 
pigment  lying  in  the  epidermis. 

20.  Cancer  is  caused  by  germs  growing  in  the  tissue. 

21.  Diptheria  can  be  controlled  by  the  use  of  Diptheria  antitoxin. 

22.  The  brain  is  separated  into  two  parts  or  hemispheres  by  a  great  longi- 
tudinal fissure. 

23.  When  oxygen  is  separated  from  other  substances  the  process  is  called  oxi- 
dation. 
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Answers  to  be 
written  here 


24.  Infectious  diseases  are  due  to  changed  methods  of  work  and  growth  on 
the  part  of  cells  in  certain  regions  of  the  body. 

25.  The  use  of  alcoholic  beverages  builds  up  the  body  and  makes  the  mus- 
cles stronger. 

26.  The  great  majority  of  grown  people  have  been  infected  with  tuberculosis 
germs, 

27.  The  sense  organs  are  the  terminations  of  the  sensory  nerves  serving  to 
carry  impressions  to  the  spinal  cord  or  brain. 

28.  Farsightedness  is  often  caused  by  a  blow  on  the  eye. 

29.  An  antiseptic  is  a  substance  which  merely  restrains  the  germs  from  grow- 
ing. 

30.  The  brain  is  in  communication  with  the  rest  of  the  body  by  means  of 
nerves. 

31.  The  cerebrum  is  the  path  of  communication  between  the  nerves  supply- 
ing the  arms,  trunk,  legs,  and  brain. 

32.  The  chief  function  of  muscles  is  to  hold  up  the  body. 

33.  All  milk  contains  bacteria. 

34.  The  alcohol  used  in  drinks  is  produced  by  the  growth  of  yeast  in  a  liquid 
containing  sugar. 

35.  Our  blood  contains  white  corpuscles  which  destroy  disease  germs. 

36.  More  people  die  daily  from  diptheria  than  from  tuberculosis. 

37.  The  use  of  tobacco  increases  the  strength  of  the  muscles. 

38.  The  use  of  tobacco  makes  the  nerve  cells  function  more  keenly. 

39.  The  chewing  of  dry  bread  aids  the  digestion  as  much  as  the  use  of  gum. 

40.  Air  is  composed  chiefly  of  two  gases,  oxygen  and  nitrogen. 

41.  The  first  step  in  treating  a  person  who  has  been  poisoned  is  to  give  an 
emetic. 

42.  Light  is  produced  by  waves  of  a  substance  called  ether. 

43.  Non-infectious  diseases  are  caused  by  small  plants  or  animals  called 
parasites  feeding  upon  the  human  body. 

44.  Alcoholic  beverages  have  great  value  in  curing  disease. 

45.  A  drink  of  alcoholic  beverage  in  the  winter  time  causes  a  man's  body  to 
become  warm. 
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Answers  to  be 
written  here 


46.  Each  portion  of  the  brain  has  its  own  definite  work  to  perform. 

47.  Fainting  is  caused  by  an  over-sufficient  supply  of  blood  being  sent  to 
the  brain. 

48.  The  spinal  cord  may  act  independently  of  the  brain  and  produce  many 
of  the  muscular  movements  necessary  in  routine  work. 

49.  The  germs  of  typhoid  fever  usually  gain  access  to  the  body  by  being 
breathed  in  with  air. 

50.  Narcotics  are  substances  which  cause  any  organs  of  the  body  to  act  more 
vigorously  than  is  their  custom. 


Answers  to  be 
written  bere 


Directions  to  teachers:  After  the  four  examples  have  been  studied  by  the  pupils,  read 
the  following  directions  to  them:  "On  the  next  page  you  will  find  a  number  of  statements 
similar  to  the  ones  you  have  just  read.  You  are  to  place  a  plus  sign  or  a  minus  sign  in  the 
column  to  the  right  of  each  statement  just  as  has  been  done  on  the  first  page.  Mark  all  of 
the  statements  that  you  are  sure  you  can  answer  correctly.  If  you  find  a  statement  that  you 
are  not  sure  you  can  answer  correctly,  study  it  carefully  and  then  mark  the  answer  you  think 
will  be  correct.  If  you  find  a  statement  you  know  nothing  about,  make  no  attempt  to  mark 
it,  as  guessing  counts  heavily  against  you.  You  will  have  25  minutes  for  the  test.  I  shall 
expect  you  to  stop  promptly  and  turn  your  folders  face  down  on  the  desk  when  I  tell  you  to 
do  so.     Ready-Go." 

In  computing  the  score  of  each  pupil  on  a  test  subtract  the  total  number  of  wrong 
answers  from  the  total  number  of  right  answers.  Such  scores  are  called  "point-scores."  In 
interpreting  them  it  is  advisable  to  form  a  distribution  which  will  show  how  many  pupils 
received  each  score.  From  the  distribution  it  is  possible  to  work  out  a  basis  for  translating 
the  point  scores  into  the  usual  kind  of  school  marks. 


Note:  The  "Directions  to  teachers"  given  above  would  not  appear  on  the  usual  printed  examinations.     They 
are  placed  here  for  the  convenience  of  teachers. 
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TRUE-FALSE  EXAMINATION  IN  HISTORY  AND  CIVIL  GOVERNMENT 

Prepared  by 

Bureau  of  Educational  Research 

University  of  Illinois 


Name Boy  or  Girl 

Age  last  birthday Next  birthday  will  be. 

Grade Date City 

School Teacher.. 


.19. 


.State. 


Below  you  will  find  a  number  of  statements.  Some  of  these  statements  are  true,  others 
are  not  true.  Read  each  statement  carefully,  then  if  it  is  true  mark  a  plus  (+)  in  the  column 
to  the  right  of  the  sentence.  If  the  statement  is  not  true  mark  a  minus  (  — )  in  the  column 
to  the  right. 

Directions  to  Teachers:  Unless  the  students  are  familiar  with  the  True- false  Examina- 
tion, explain  the  procedure  to  them  by  using  four  examples  as  given  in  the  Examination 
in  Physiology,  Appendix,  p.  44;  Time  allowance  for  this  examination,  37  minutes.  See 
also  Directions  to  teachers,  Appendix,  p.  47. 


1.  During  Jackson's  time  the  national  convention  method  of  nominating 
presidential  candidates  was  introduced. 

2.  The  Constitution  was  gladly  accepted  by  all  of  the  people. 

3.  Jefferson  was  the  second  president  of  the  U.  S. 

4.  John  C.  Calhoun  was  a  strong  advocate  of  the  doctrine  of  "Nullifica- 
tion." 

5.  Throughout  the  Revolutionary  War  the  American  navy  was  superior  to 
that  of  England. 

6.  The  reconstruction  period  following  the  World  War  was  a  very  trouble- 
some one  in  the  U.  S. 

7.  During  the  first  two  years  of  the  Civil  War  the  confederate  troops  won 
the  majority  of  the  victories. 

8.  England  removed  all  the  taxes  against  which  the  colonists  rebelled. 


Answers   to  be 
written  here 
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9.     Scientific  farming  in  the  U.  S.  was  little  thought  of  before  1860. 

10.  France  entered  the  World  War  to  save  Russia. 

11.  Robert  E.  Lee,  leader  of  the  Confederate  forces,  was  one  of  the  greatest 
soldiers  of  the  Civil  Wrar. 

12.  In  the  war  of  1793  between  France  and  England,  the  U.  S.  sided  with 
France. 

13.  In  the  battle  with  the  English  warship  "Serapis,"  the  flagship  of  Paul 
Jones,  the  "Bon  Homme  Richard,"  was  completely  destroyed. 

14.  President  Hayes  was  elected  over  his  rival  candidate  by  an  overwhelming 
majority. 

15.  The  Federal  Reserve  Act  of  1913  exercises  an  important  effect  in  pre- 
venting money  crises. 

16.  During  Monroe's  presidency  there  was  more  internal  strife  and  turmoil 
than  during  any  previous  period  since  the  Revolution. 

17.  John  Brown's  Raid  at  Harper's  Ferry  brought  many  non-slave-holders 
to  the  side  of  the  slave-holders. 

18.  Russia  entered  the  World  War  to  protect  Serbia. 

19.  The  Lincoln-Douglas  debates  won  Lincoln  the  Illinois  senatorship  over 
Douglas. 

20.  In  ten  years  the  tolls  and  revenues  from  the  Erie  Canal  repaid  the  entire 
amount  spent  during  the  eight  years  of  its  construction. 

21.  During  the  period  of  reconstruction  following  the  Civil  War,  Congress 
passed  laws  prohibiting  the  negroes  from  voting. 

22.  The  issue  of  Wilson's  second  campaign  was  the  keeping  of  the  U.  S.  out 
of  war. 

23.  At  the  end  of  the  Spanish-American  war,  Spain  turned  Cuba  over  to  the 
U.  S.  as  a  colony  and  paid  us  $20,000,000. 

24.  Many  important  inventions  were  made  between  1830-1860. 

25.  Washington  Irving  was  the  first  great  American  poet. 

26.  During  the  revolution  the  population  of  the  United  States  increased. 

27.  John  J.  Pershing,  U.  S.  General,  was  put  in  supreme  command  of  all  of 
the  allied  forces  during  the  World  War. 

28.  Andrew  Jackson  was  one  of  the  most  cultured  of  our  presidents. 

29.  The  Missouri  Compromise  prevented  slavery  in  all  of  the  Louisiana 
territory  north  of  36°  30',  except  in  Missouri. 
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Answers  to  be 
written   here 


30.  At  the  battle  of  Manila  Bay,  a  German  fleet  would  have  helped  the 
Spaniards  had  not  the  Admiral  of  a  British  fleet  warned  the  Germans  not 
to  interfere. 

31.  The  Dred  Scott  Decision  meant  that  negroes  could  not  become  citizens. 

32.  England  entered  the  World  War  because  the  German  fleet  bombarded 
English  coastal  towns. 

33.  In  tilling  the  soil,  the  farmers  at  first  were  afraid  to  use  a  cast-iron  plow, 
saying  that  it  poisoned  the  soil. 

34.  The  English  government  sent  American  cotton  manufacturers  copies  of 
the  spinning  jenny  soon  after  its  invention. 

35.  In  the  final  battle  of  the  Texas  rebellion  against  Mexico,  the  Mexicans 
were  outnumbered  two  to  one. 

36.  The  early  colonists  believed  in  witchcraft. 

37.  General  Braddock's  army  saved  Washington  and  his  Virginian  troops 
from  an  Indian  massacre. 


Answers  to  be 
written  here 


Georgia  was  populated  by  people  who  had  been  thrown  into  jail  for 
debt. 

William  Pitt,  as  prime  minister  of  England,  was  a  great  aid  to  the  colon- 
ists in  driving  the  French  from  the  Atlantic  coastal  region. 


41.  Lincoln  was  killed  in  a  battle  during  the  Civil  War. 

42.  Washington  belonged  to  the  Federalist  party. 

43.  The  Referendum  is  a  process  of  obtaining  legislation  which  the  people 
desire,  in  spite  of  an  unwilling  legislature. 

44.  From  1830-1860  the  population  of  the  U.  S.  doubled. 

45.  John  Adams  was  the  first  president  to  be  inaugurated  in  the  city  of 
Washington. 

46.  During  the  colonial  period  books  were  plentiful. 

47.  The  British  encouraged  the  Indians  to  kill  American  settlers  who  moved 
into  the  Northwest  Territory. 

48.  Manufacturing  was  the  important  industry  from  the  time  of  the  Revolu- 
tion down  to  Jackson's  time. 

49.  The  battle  of  Gettysburg  was  the  only  battle  of  the  Civil  War  fought  on 
northern  soil. 


50 


50.  The  Initiative  is  a  process  of  preventing  legislation  which  a  legislature 
would  impose  upon  the  people. 

51.  James  Fenimore  Cooper  was  the  first  American  author  to  receive  recogni- 
tion in  Europe. 

52.  The  South,  after  all  of  the  states  had  seceded,  had  more  men  for  soldiers 
than  the  North. 

53.  England  was  not  as  kind  to  her  colonies  as  the  other  European  nations. 

54.  The  battle  ship  Maine  was  sunk  in  the  harbor  of  Havana  by  a  torpedo 
fired  from  a  Spanish  warship. 

55.  At  the  close  of  the  Mexican  war,  Mexico  was  forced  to  pay  the  U.  S. 
$100,000,000  and  to  pay  claims  of  $3,500,000  which  American  citizens 
had  against  her. 

56.  The  presidential  message  sometimes  has  great  influence  upon  Congress. 

57.  During  the  administration  of  President  Hayes  much  bitter  feeling  arose 
between  the  North  and  the  South. 

58.  Lincoln  accepted  the  presidency  with  the  determination  to  forever  wipe 
out  slavery. 

59.  The  annexation  of  Texas  led  directly  to  war  with  Mexico. 

60.  William  Lloyd  Garrison  was  a  violent  anti-abolitionist. 

61.  During  the  period  from  1750-1800  the  steamboat  was  the  main  means  of 
travel  where  streams  were  available. 

62.  Many  northern  people  opposed  Lincoln's  reelection. 

63.  John  Adams  was  the  second  president  of  the  U.  S. 

64.  One  result  of  the  war  of  1812  was  to  stimulate  manufacturing  in  New 
England. 

65.  Madison  was  very  much  opposed  to  Jefferson's  policies. 

66.  The  battle  of  Gettysburg  was  the  turning  point  in  the  Civil  War. 

67.  The  spoils  system  has  been  very  beneficial  to  this  country. 

68.  Jay's  treaty  with  England  was  much  opposed  in  the  U.  S. 

69.  The  underground  railroad  was  a  railroad  used  to  carry  cotton  from  the 
south  to  the  mills  of  New  England. 

70.  John  Quincy  Adams  possessed  the  characteristics  of  an  ideal  president. 

71.  The  beginning  of  the  westward  movement  was  during  the  administra- 
tion of  John  Quincy  Adams. 

72.  Washington  was  inaugurated  in  New  York  City. 

73.  During  the  latter  part  of  the  18th  century  the  U.  S.  was  carrying  on 
trade  with  China. 

74.  The  war  with  Mexico  lasted  over  three  years. 

75.  Lincoln  declared  war  on  the  southern  states  to  make  them  give  up 
slavery. 
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TRUE-FALSE  EXAMINATION  IN  GEOGRAPHY 

Prepared  by 

Bureau  of  Educational  Research 

University  of  Illinois 


Name Boy  or  girl 

Age  last  birthday Next  birthday  will  be 

Grade Date City State. 

School Teacher 


.19. 


Below  you  will  find  a  number  of  statements.  Some  of  these  statements  are  true,  others 
are  not  true.  Read  each  statement  carefully,  then  if  it  is  true  mark  a  plus  (+)  in  the  column 
to  the  right  of  the  sentence.  If  the  statement  is  not  true  mark  a  minus  (-)  in  the  column 
to  the  right. 

Directions  to  teachers:  Unless  the  students  are  familiar  with  the  True-False  Examina- 
tion, explain  the  procedure  to  them  by  using  four  examples  as  given  in  the  Examination  in 
Physiology,  Appendix,  p.  44.  Time  allowance  for  this  examination,  40  minutes.  See  also 
Directions  to  teachers,  Appendix,  p.  47 


1.  Agriculture  is  the  principal  industry  of  France. 

2.  Every  portion  of  South  America  has  a  very  warm  climate  during  all 
seasons  of  the  year. 

3.  The  lumber  resources  of  India  have  been  little  developed. 

4.  Very  little  rain  falls  in  the  vicinity  of  the  equator. 

5.  In  the  forests  of  South  America  are  found  many  fierce,  wild  animals. 

6.  China  is  governed  by  a  republican  form  of  government. 

7.  Trees  prevent  rain  water  from  running  rapidly  off  the  land. 

8.  The  main  industry  of  Australia  is  manufacturing. 

9.  Russia  in  Europe  has  many  good  harbors. 

10.  Many  Europeans  have  settled  in  Central  Africa. 

11.  Switzerland  has  large  coal  and  mineral  deposits. 

12.  Argentina  is  one  of  the  more  backward  of  South  American  countries. 
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13.  There  is  not  a  great  variation  in  the  amount  of  rainfall  which  is  received 
by  different  sections  of  Europe  in  a  year. 

14.  People  who  live  in  a  warm  climate  are  not  as  industrious  as  those  who 
live  in  a  temperate  climate. 

15.  At  the  time  of  the  arrival  of  Columbus  the  natives  of  South  America 
were  all  in  the  lower  stages  of  barbarism. 

16.  Manufacturing  is  an  important  industry  in  Germany. 

17.  When  Columbus  discovered  South  America  it  was  inhabited  by  both  red 
and  white  men. 

18.  London  is  the  largest  city  in  the  world. 

19.  The  coast  of  Australia  is  very  regular. 

20.  The  population  of  Great  Britain  is  nearly  one-half  as  great  as  that  of  the 
entire  U.  S. 

21.  Africa  has  two  deserts,  one  north  and  one  south  of  the  equator. 

22.  Belgium,  though  smaller,  has  more  people  than  Holland. 

23.  France  produces  more  coal  than  the  British  Isles. 

24.  Much  of  the  Netherlands  lies  below  sea  level. 

25.  The  work  done  by  the  Great  Ice  Sheet  was  a  hindrance  to  the  further  de  - 
velopment  of  our  country. 

26.  Very  little  mining  is  carried  on  in  Spain  and  Portugal. 

27.  China  has  the  largest  number  of  inhabitants  of  any  country  in  the  world. 

28.  Before  the  time  of  Christ,  Greece  was  one  of  the  most  backward  and 
barbaric  countries  of  the  world. 

29.  Asia  has  much  the  same  climate  throughout. 

30.  Corn  is  the  principal  food  of  the  Japanese  people. 

31.  The  Brazilian  forests  produce  many  valuable  products. 

32.  The  Mississippi  Valley  is  the  most  extensive  farming  section  in  the 
United  States. 

33.  Italy  has  many  very  valuable  mineral  deposits. 

34.  The  plains  of  Russia  are  populated  by  many  different  peoples. 

35.  Agriculture  is  the  principal  industry  of  Belgium. 

36.  In  early  times  the  Chinese  led  the  world  in  the  matter  of  civilization. 
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37.  In  early  historical  times,  the  Mississippi  river  was  longer  than  it  is  now. 

38.  The  people  of  Spain  and  Portugal  are  very  backward. 

39.  The  majority  of  the  Asiatic  people  belong  to  the  yellow  race. 

40.  Europe  has  the  most  regular  coast  of  all  the  continents. 

41.  All  of  Australia  has  a  very  heavy  rainfall. 

42.  Chile  is  one  of  the  great  copper-producing  countries  of  the  world. 

43.  Manufacturing  is  the  principal  industry  of  South  Africa. 

44.  China  has  more  inhabitants  than  all  European  countries  taken  together. 

45.  Vegetable  and  fruit  growing  is  extensively  followed  on  the  Coastal  Plains. 

46.  Brazil  is  smaller  than  the  United  States. 

47.  Agriculture  is  the  principal  industry  of  Switzerland. 

48.  All  of  the  territory  covered  by  Spain  and  Portugal  has  abundant  rainfall. 

49.  The  French,  attracted  by  the  discovery  of  rich  deposits  of  gold  and  silver, 
seized  almost  all  of  South  America. 

50.  Our  present  population  is  over  150,000,000. 

51.  The  principal  industry  of  Venezuela  is  mining. 

52.  Very  few  forests  are  found  in  European  Russia. 

53.  Some  parts  of  the  German  Empire  are  too  arid  for  agriculture. 

54.  South  America  is  quite  like  North  America  in  its  surface  features. 

55.  Many  manufacturing  industries  have  been  established  along  the  western 
coast  of  the  United  States. 

56.  Africa  is  governed  by  many  different  nations. 

57.  Chile  is  one  of  the  least  progressive  nations  in  S.  A. 

58.  Great  Britain  has  more  manufacturing  than  any  other  nation. 

59.  Europe  should  not  be  called  a  continent. 

60.  Very  little  agriculture  is  carried  on  in  Argentina. 

61.  Much  of  the  coast  of  South  America  has  been  sinking. 

62.  Australia  is  larger  than  the  U.  S. 

63.  Irrigation  is  little  practised  in  the  western  states. 

64.  Great  Britain  has  a  greater  foreign  trade  than  any  other  nation  of  the 
earth. 
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65.  Very  little  manufacturing  is  carried  on  in  Greece. 

66.  Little  agriculture  is  carried  on  in  Uruguay  and  Paraguay. 

67.  The  discovery  of  gold  in  Australia  led  to  the  development  of  that 
country's  resources. 

68.  Commerce  in  Holland  is  highly  developed. 

69.  Irrigation  is  extensively  developed  in  India. 

70.  Agriculture  is  the  principal  industry  in  Italy. 

71.  Most  of  Russia  is  a  level  plain. 

72.  Egypt  is  now  governed  by  the  Turks. 

73.  The  Chinese  cities  are  very  crowded. 

74.  Most  of  the  Asiatic  people  are  either  Brahmans  or  Buddhists. 

75.  The  government  of  Great  Britain  is  a  limited  monarchy. 

76.  The  Nile  river  flows  across  the  desert. 

77.  The  Delaware  Water  Gap  is  in  Pennsylvania. 

78.  Commercially,  Italy  occupies  a  very  favorable  position. 

79.  Many  more  animals  have  their  homes  on  land  than  in  the  sea. 

80.  The  principal  occupation  of  Greece  is  coal  mining. 

81.  Grain  farming  is  the  principal  industry  of  Rumania  and  Bulgaria. 

82.  More  than  half  of  Europe  is  an  extensive  lowland. 

83.  More  than  one-third  of  our  population  is  made  up  of  foreigners  and 
and  negroes. 

84.  Animal  life  is  found  in  great  abundance  throughout  South  America. 

85.  Ireland  is  mainly  a  country  of  farms. 

86.  The  roads  and  railways  of  Switzerland  are  very  poor. 

87.  The  population  of  North  America  exceeds  200,000,000  people. 

88.  The  Sahara  desert  is  practically  as  large  as  the  U.  S. 

89.  Wheat  is  the  main  crop  in  the  Valley  of  the  Red  River  of  the  North. 

90.  Mount  Vesuvius,  in  Italy,  is  constantly  in  eruption. 

91.  In  some  places  the  Amazon  river  is  several  miles  wide. 

92.  The  Japanese  people  have  been  very  quick  to  learn  the  lessons  of  west- 
ern civilization. 
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93.  Uruguay  and  Paraguay  are  very  mountainous  countries. 

94.  Very  little  manufacturing  is  carried  on  in  the  Middle  Atlantic  States. 

95.  The  Tropical  Andean  Countries  are  among  the  most  important  mineral 
producing  regions  of  the  world. 

96.  The  principal  food  of  the  Chinese  is  corn. 

97.  All  of  Africa  is  populated  by  the  negroes. 

98.  Much  of  the  surface  of  Japan  is  mountainous. 

99.  The  people  of  Asia  have  always  been  very  backward. 

100.  The  government  of  Japan  is  an  absolute  monarchy. 
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COMPLETION  EXAMINATION  IN  AMERICAN  GOVERNMENT 

Prepared  by 
Bureau  of  Educational  Research 
University  of  Illinois 


Name Boy  or  Girl 

Age  last  birthday Next  birthday  will  be 19. 

Grade Date City State 

School Teacher 


Below  you  will  find  a  number  of  statements.  In  each  statement  one  or  more  important 
words  have  been  omitted.  Each  blank  in  the  sentence  shows  where  a  word  has  been  left  out. 
Read  each  statement  carefully,  then  write  in  the  blank  the  word  which  completes  the  meaning 
of  the  statement.     You  will  be  allowed  15  minutes  for  the  test. 

1.  The  primary  purpose  for  which  government  exists  is  the of  our  lives 

and  property. 

2.  Citizenship  may  be  acquired  by in  this  country  or  by  a  process  of 

for  natives  of  other  lands. 

3.  Our  national  government  derives  its  authority  from  the of  the  United 

States  through  our  national 

4.  The  legislative  power  granted  to  the  national  government  is  vested  in  a  Congress  of 

houses,  the  smaller  of  which  is  called  the and  the 

larger  the 

5.  The  execution  of  the  laws  made  by is  intrusted  to  the 

of  the  United  States. 

6.  All  judges  connected  with  the  national  courts  are  appointed  for  life  with  the  consent 
of. 

7.  Most  of  the  candidates  for  office  which  are  filled  by  popular  vote  are  nominated  directly 
in 

8.  The  Fifteenth  Amendment  of  the  United  States  Constitution  prevents  the  states  from 

denying  to  citizens  the  right  to  vote  on  account  of , 

,  or  previous  condition  of 
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9.     Practically  all  of  our  law-making  bodies  are  made  up  of. chosen 

for  short  terms  from into  which  the  states,  counties  and  cities  are 

divided. 

10.  The  first  permanent  English  settlements  in  America  were  made  in  what  is  now  the  state 
of. 

11.  In  a  county,  the  records  of  the  county  board  and  other  official  papers  are  preserved  by 
the  county 

12.  All  cities  are  public  corporations  created  under municipal  laws. 

13.  Every  incorporated  city  obtains  from  the government  a 

under  which  it  may  elect  its  officials  and  conduct  its  business. 

14.  Civil  service  employees  may  be  removed  from  service  only  for 

15.  The  power  of  impeaching  a  state  officer  is  given  to  the 

16.  The is  by  far  the  most  prominent  and  powerful  executive  official 

in  the  state.     Very state  officers  are  appointed  by  him  or  are  responsi- 
ble to  him. 

17.  All  important  officials  connected  with  the  executive  or  judicial  service  of  the  United 

States  may  be  removed  by through  the  lower  house  of  Congress 

and  by in  the  senate. 

18.  Far  more  property  is  destroyed  by than  by  all  other  agencies. 

19.  There  is  no  task  of  state  and  local  government  which  outranks  in  importance  that  of 
providing  an education  at  public  expense. 

20.  All  rivers  and  canals  within  a  single  state  are  controlled  by  the 

in  which  they  are  located. 

21.  Most  of  the  revenue  for  state  and  local  governments  is  secured  by  a 

on 

22.  A  state is  the  fundamental  law  which  the  people  of  the  state  have 

arranged  for  their  government  and  protection. 

23.  A  state  constitution  can  be  changed  by  means  of  an 

24.  The  three-fifths  compromise  provided  that  five should  be  counted  as 

equal   to   three when   reckoning   the for  either 

direct  taxation  or  representation. 
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RECOGNITION  EXAMINATION  IN  ALGEBRA 

Prepared  by 

Bureau  of  Educational  Research 

University  of  Illinois 


Name Boy  or  girl 

Age  last  birthday Next  birthday  will  be 19.. 

Grade Date City State 

School Teacher 


Below  you  will  find  a  number  of  statements.  In  each  statement  a  word  or  number  has 
been  omitted.  At  the  close  of  the  statement  several  words  or  numbers  have  been  given. 
One  of  these  is  the  correct  answer.  Select  the  word  or  number  which  you  think  is  correct  and 
draw  a  line  under  it.  Most,  if  not  all,  of  the  examples  can  be  solved  by  mental  calculation. 
If  any  figuring  is  necessary,  work  on  the  margin  of  the  page.  You  will  be  allowed  17  minutes 
for  the  test. 

1.  Numbers  that  are  represented  by  letters  are  called numbers. 

substituted — literal 

2.  When  two  or  more  letters  are  multiplied  together  each  is  called  a 

of  the  product.  factor — coefficient 


3.  If  a  man  rides  a  certain  distance  in  10  hours,  in  h  hours  he  rides 

,m.    h      10 
10h;—  ;  — 

10      h 

4.  The  statement  2x+5=29  is  called  an identity — equation. 

5.  If  16  is  substracted  from  three  times  a  certain  number  the  result  is  110.     The  number 

is 36  2/3;  31  1/3;  42. 

6.  A  number  which  is  a  factor  of  two  or  more  numbers  is  called  a factor. 

common — equal 

7.  If  there  are  two  equal  factors  of  a  number,  either  is  called  the of  the 

number.  square  root — common  factor. 

8.  To  multiply  algebraic  fractions  take  the of  the  numerators  for  a 

new  numerator  and  the  product  of  the  denominators  for  a  new  denominator. 

sum — product 
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9.     A  fraction  whose  numerator  or  denominator  (or  both)  contains  fractions  is  called  a 
fraction.  multiple — complex 

10.  A  is  a  statement  of  a  fact  which  is  to  be  proved,     theorem — axiom. 

11.  The  name  given  the  +  sign  is negative — positive 

12.  To  find  the  sum  of  two  numbers  whose  signs  are  opposite,  take  their 

regarding  each  as  positive,  and  prefix  the  sign  of  the  larger  number  to  the  answer. 

sum — difference — product 

13.  Whenever  a  number  occurs  without  a  sign,  the sign  is  to  be  under- 

stood.  X;  +;  — 


14.  The  number  denoting  the  power  of  a  term  is  called  the 

prefix — exponent 

2abc 

15.  Ifa=2,b  =  -3andc  =  -5  then    = -6;  -2;  30 

a 

16.  In  adding  like  terms  add  the  coefficients  for  the  new  coefficient  and 

it  by  the  common  factor.  multiply — divide. 

17.  An  expression  which  contains  more  than  one  term  is  called  a 

monomial — polynomial 

1 8.  If  the  length  of  a  rectangle  =  4  feet  more  than  twice  the  width,  the  perimeter  =  56  feet. 
The  length  =  feet.  8—12—16 

19.  Any  term  may  be  transposed  from  one  side  of  an  equation  to  the  other,  provided  its 
is  changed.  sign — value. 

20.  Any  equation  which  contains  no  higher  power  of  the  unknown  letter  than  the  first  is 
called     a equation.  radical — simple. 

21.  The  exponent  of  the  product  of  two  powers  of  the  same  number  is  equal  to  the 
of  the  exponents  of  the  factors.  product — sum 

22.  To  raise  the  product  of  two  numbers  to  any  power,  raise  the  numbers  separately  to  that 
power  and  take  their product — sum 

23.  The  square  of  any  two  numbers  is  equal  to  the  square  of  the  first  number 

twice  the  product  of  the  two  plus  the  square  of  the  second  number,     plus — minus 

24.  A  20  foot  ladder  rests  against  a  building,  the  bottom  of  the  ladder  being  12  feet  from  the 
cellar  wall.     The  top  is feet  from  the  ground.  8-i6 

25.  In  division,  the  sign  of  the  quotient  is whenever  the  dividend  and  di- 
visor have  like  signs.                     — ;  + 
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26.  In  finding  the  quotient  of  two  powers  of  the  same  number  the  exponent  of  the  quotient 

is  equal  to  the  exponent  of  the  dividend by  that  of  the  divisor. 

increased — diminished. 

27.  (3x2-2x-l)-Kx-l)= 3x+l;3x-l. 

28.  A  factor  which  has  no  factor  except  itself  and  unity  is  called  a factor. 

prime — multiple 

29.  The  product  of  all  the  common  prime  factors  of  two  or  more  numbers  or  expressions  is 
called  their common  factor.  highest — lowest. 

30.  If  one  number  is  exactly  divisible  by  another,  the  first  is  called  a of  the 

second.  Divisor — multiple 

31.  In  algebraic  fractions  the  dividend  is  called  the 

denominator — numerator 
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EXAMPLES  OF  TRADITIONAL  EXAMINATIONS 

At  the  request  of  the  Director  of  the  Bureau  of  Educational  Research,  a  number  of 
superintendents  and  high-school  principals  in  Illinois  sent  copies  of  final  examinations  which 
were  being  given  in  their  schools  during  the  year  1921-1922.  The  following  list  of  questions 
has  been  selected  from  the  large  number  submitted.  These  questions,  although  the  selection 
was  made  on  the  basis  of  the  judgment  of  one  person  only,  seem  among  the  best  examples 
of  the  traditional  type  of  examination.  In  that  they  demand  explanation,  discussion,  organi- 
zation of  material,  exercise  of  judgment,  etc.,  on  the  part  of  the  pupil,  they  call  for  a  distinct 
type  of  ability  not  required  by  any  form  of  the  "new  examination."  In  making  this  selection 
the  different  school  subjects  and  the  different  grades  both  of  the  elementary  and  of  the  high 
school  have  been  represented  to  as  great  a  degree  as  possible.  In  the  lower  grades  of  the 
elementary  school  the  majority  of  the  questions  submitted  were  based  upon  memory  and 
could  easily  have  been  changed  into  some  form  of  the  "new  examination."  For  that  reason, 
no  examinations  from  those  grades  are  included  in  this  selection. 

HISTORY 
(Seventh  grade) 

1.  What  was  the  purpose  of  Columbus'  voyage  and  its  result? 

2.  What  did  ten  of  the  following  explore  or  discover:     Cabot,  Balboa,  Ponce  de  Leon, 
Magellan,  Coronado,  De  Soto,  Drake,  Hudson,  Cartier,  Champlain,  LaSalle. 

3.  What  people  settled  Jamestown,  Virginia?     Discuss  one  of  the  following  topics  in  con- 
nection with  Virginia  history. 

The  Starving  Time,  Individual  Ownership,  Tobacco  Raising. 

4.  Tell  by  what  class  of  people  and  why  was  each  of  the  following  states  settled:     Maryland, 
Carolina,  Georgia. 

5.  Tell  the  story  of  William  Penn,  naming  the  colony  he  founded. 

6.  Why  did  the  Pilgrims  come  to  America?     Where  did  they  land?     Write  of  their  first 
winter  in  America  and  their  relations  with  the  Indians. 

7.  Tell  what  happened  in  1492,  1607,  1619,  1620,  from  1519  to  1522. 

8.  For  what  nation  and  on  what  errand  did  Joliet  come  to  the  Illinois  country?     Why  did 
Marquette  come  with  him? 

9.  Tell  the  story  of  Starved  Rock.     What  tribes  of  Indians  were  connected  with  it  and  how? 

10.  Write  an  item  of  historical  interest  about  each  of  five:    Tonti,  Stuyvesant,  Oglethorpe, 
Baltimore,  Bacon,  Bradford. 

11.  Write  a  short  paragraph  about  two  of  the  following: 

1.  Illinois  pioneers. 

2.  Illinois  rangers. 

3.  Block  houses. 

4.  Keel  boats. 
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HISTORY 

(Seventh  grade) 

1.  What  were  the  results  of  the  Revolutionary  War? 

2.  Explain  in  what  ways  Congress  was  weak  under  the  Articles  of  Confederation. 

3.  Who  was  Lafayette?    What  did  he  do  for  American  liberty?    Why? 

4.  What  was  the  Ordinance  of  1787? 

5.  A  convention  of  delegates  from  the  states  was  called  to  meet  in  Philadelphia  in  May, 
1787.     Why? 

6.  Who  was  the  first  President  of  the  U.  S.  under  the  Constitution?  When  and  where  was 
he  inaugurated? 

7.  Name  and  tell  how  important  inventions  have  helped  the  progress  of  the  U.  S. 

8.  Why  was  the  purchase  of  Louisiana  an  important  event  for  the  U.  S? 

9.  What  was  the  result  of  the  War  of  1812? 

10.     Tell  something  of  the  work  of  the  Humanitarians  and  the  establishment  of  the  free  ele- 
mentary schools. 

HISTORY 

(Eighth  grade) 

1.  Discuss  five  powers  or  duties  of  Congress. 

2.  Name  the  officers  of  the  President's  cabinet  and  a  duty  of  each. 

3.  Explain  the  need  of  a  survey  system.  Make  a  diagram  showing  baseline,  principal 
meridian,  township  lines,  range  of  townships.     Locate  Twp.  2  N.  R.  3  E  of  P.  M. 

4.  How  were  ten  of  the  following  connected  with  the  Civil  War:  Stonewall  Jackson,  Major 
Anderson,  Robert  E.  Lee,  Jefferson  Davis,  McClellan,  Hooker,  Grant,  Sherman,  Emanci- 
pation Proclamation,  Hammering  Campaign,  Gettysburg,  and  Appomattox  Court  House. 

5.  Tell  the  location,  time,  inventor,  and  importance  of  the  Atlantic  Cable. 

6.  Tell  of  two  good  results  of  the  Civil  Service  Reform. 

7.  Name  and  discuss  two  famous  laws  we  have  studied  this  semester. 

8.  Write  a  brief  paragraph  discussing  the  importances  of  the  Pan-American  Congresses. 

9.  Give  the  time,  place,  purpose  and  importance  of  an  exposition  studied  this  semester. 

10.  State  two  causes  and  two  results  of  the  Spanish-American  War. 

11.  Write  a  statement  about  each  in  connection  with  the  World  War:  Autocracy;  "der  Tag;' 
submarine;  Lusitania;  armistice. 

HISTORY 

(High  school) 

1.  Discuss  the  work  of  Spain  in  exploration — naming  five  important  explorers. 

2.  State  the  Mercantile  Theory  of  trade  and  explain  its  effects  during  the  American  colonial 
period. 
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3.  State  five  defects  in  the  Articles  of  Confederation.     How  were  these  defects  remedied 
in  the  Constitution? 

4.  What  was  the  Northwest  Ordinance  of  1787?     Why  important? 

5.  Explain  Alexander  Hamilton's  policy  on  the  U.  S.  debt. 

6.  Give  the  history  of  the  Nullification  Controversy  of  1828-33. 

7.  Give  the  history  of  the  election  of  1823. 

8.  Identify  the  following:     Gallatin;  Oglethorpe;  DeWitt  Clinton;  Thos.  Paine;  Stephen 
Decatur. 

9.  Explain  three  important  results  of  the  war  of  1812. 

10.     Discuss  the  political  platform  and  policies  of  President  Jefferson. 


ANCIENT  HISTORY 

(High  school) 

1.  (a)     Name  and  explain  the  sources  of  historical  information. 

(b)     Name  each  of  the  Oriental  nations  in  the  order  of  their  development.     State  what 
was  done  by  each  for  civilization. 

2.  (a)     What  were  the  causes  of  Greek  colonization?      What  relation  did  the  Greek  colony 

have  to  the  mother  city? 
(b)     Locate  the  chief  centers  of  colonization  and  state  for  what  each  was  famous. 

3.  (a)     Trace  out  the  history  of  the  ancient  Hebrew  people  and  explain  their  service  to 

humanity. 
(b)     Describe  the  government  and  customs  of  the  Spartans. 

4.  (a)     Explain  the  origin,  growth,  and  effect  of  the  Delian  Confederacy  on  Greek  history. 

Show  how  it  changed  into  the  Athenian  Empire. 

(b)  Describe  Athens  in  the  time  of  Pericles. 

(c)  Describe  the  intellectual  greatness  of  Athens  in  the  time  of  Pericles. 

5.  (a)     Identify  the  following  men  and  account  for  their  greatness. 

1.  Alexander  3.     Aristides 

2.  Themistocles  4.     Plato 

5.     Socrates. 

(b)     Give  an  account  of  the  invasion  of  Greece  by  Xerxes  during  the  Persian  Wars. 
Name  important  battles  and  tell  in  detail  about  one. 

6.  (a)     What  did  Philip  of  Macedon  accomplish  for  Macedonia? 
(b)     Trace  the  march  of  Alexander  against  the  Persians. 

7.  (a)     Why  was  Europe  better  fitted  than  Asia  to  develop  the  highest  civilization? 

(b)     What  mountain  systems  of  Europe  are  not  off-shoots  from  the  central  mass  of  the 
Alps? 
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GRAMMAR 

(Eighth  grade) 

Answer  ten  questions. 

1.  Of  what  value  is  a  good  vocabulary? 

2.  How  can  a  person  acquire  a  good  command  of  words? 

3.  What  is  an  antonym?     Write  five  words  and  give  their  antonyms. 

4.  Define  synonym.     Write  five  words  and  give  their  synonyms. 

5.  Define  verb.     A  verb  phrase. 

6.  Write  a  stanza  of  the  Star  Spangled  Banner.     Punctuate  correctly  and  underline  the 
verbs  and  verb  phrases. 

7.  Define  transitive  verb.     A  direct  object.     Write  a  sentence  containing  a  transitive 
verb  and  a  direct  object. 

8.  Name  the  eight  parts  of  speech. 

9.  A  boy  is  flying  a  kite. 

A  crow  is  flying  over  the  cornfield. 

Are  the  verbs  in  the  above  sentences  transitive  or  intransitive?     Why? 

10.  Use  the  verbs,  lie  and  lay,  sit  and  set,  correctly  in  sentences. 

11.  Classify  the  nouns  and  verbs  as  to  number  in  the  following: 

a.  They  came  e.  The  house  will  be  built. 

b.  We  come  f.  The  horses  were  running 

c.  We  have  come  g.  They  have  been  seen 

d.  We  had  come  h.  We  saw 

12.  What  is  a  participle?     Underline  and  give  the  grammatical  construction  of  the  participle 
in  the  following  sentence: 

Crossing  the  street,  I  lost  my  hat. 

13.  What  are  the  distinguishing  marks  of  a  verbal  noun? 

14.  Give  the  construction  of  the  verbal  nouns  in  the  following  sentences: 

a.  To  obey  is  a  cardinal  virtue. 

b.  Most  boys  like  to  play  basketball. 

c.  Playing  baseball  is  hard  work. 

d.  I  enjoy  hearing  pupils  read. 

15.  Name  and  define  the  tenses.     Tabulate  the  tenses  of  the  verb  "call." 

ENGLISH 

(High  school,  1st  year) 

1.     (a)  In  comparison  with  other  languages  is  English  old  or  new? 

(b)  Why  is  the  study  of  Latin  important  to  us? 

(c)  Give  an  example  of  a  word  derived  from  Latin  and  explain  its  parts. 

(d)  Explain  by  illustration  the  use  of  a  prefix. 

(e)  Define  literally:     irregular — international. 
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2.  (a)  Explain  the  value  of  good  pronunciation. 

(b)  List  5  words  that  you  have  been  mispronouncing. 

(c)  Punctuation  aids  one  in  what  way? 

(d)  Illustrate  the  use  of  one  punctuation  mark. 

(e)  State  and  illustrate  the  simple  rule  of  spelling  in  regard  to  ei  and  ie. 

3.  (a)     Name  the  8  parts  of  speech.     Illustrate. 

(b)  Why  should  one  make  a  good  choice  of  words  in  writing  or  speaking? 

(c)  Write  an  exclamatory  sentence. 

(d)  Give  the  plurals  of:  datum,  radius,  lady,  alumna,  monkey,  index,  oasis,  cargo,  soloy 
volcano. 

(e)  Give  feminine  of:  abbot,  hero,  wizard,  sir,  lord. 

4.  (a)     How  would  you  distinguish  poetry  from  prose? 

(b)  Give  an  example  of  rhyme. 

(c)  Name  3  poetic  qualities  of  Ancient  Mariner?     Illustrate. 

(d)  Quote  your  favorite  stanza  from  the  poem,  Ancient  Mariner. 

(e)  Name  two  descriptive  passages  from  Vision  of  Sir  Launfal. 

5.  (a)     What  is  the  purpose  of  the  drama? 

(b)  Name  and  give  the  dates  of  our  greatest  English  dramatist. 

(c)  Is  "The  Merchant"  tragedy  or  comedy?     Why? 

(d)  Name  the  4  stories  which  make  up  the  plot. 

(c)  Quote  3  passages  from  the  play. 

(d)  What  is  the  climax  of  the  play?     Explain  why. 

(e)  Give  setting  of  the  story.     Its  source. 

6.  (a)     Describe  the  theatre  of  Shakespeare's  time,  or  characterize  Shylock  carefully, 

illustrating  your  points. 

ENGLISH 

(High  school) 

Answer  ten  questions. 

1.  (a)     Discuss  the  work  of  three  colonial  prose  writers, 
(b)     Discuss  the  work  of  two  colonial  poets. 

2.  (a)     What  were  the  general  tendencies  of  the  literature  of  the  Revolution? 

(b)     Discuss  the  works  of  two  writers  who  were  closely  connected  with  governmental 
affairs. 

3.  Who  was  the  first  American  novelist?    Tell  about  his  works  and  characteristics  as  a 

writer. 

4.  (a)     Give  five  important  facts  concerning  the  life  of  Irving, 
(b)     Write  in  outline  form  a  classification  of  Irving's  works. 

5.  Give  a  detailed  account  of  the  life  of  your  favorite  American  poet. 

6.  (a)     Name  three  striking  characteristics  of  the  poetry  of  each  of  the  six  great  American 

poets, 
(b)     Name  two  of  the  best  poems  of  each. 

7.  (a)     Give  five  facts  concerning  the  life  of  Poe. 

(b)     Give  ten  striking  characteristics  of  his  work. 
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8.  (a)     Discuss  the  prose  of  Emerson,  Lowell,  and  Holmes, 
(b)     Quote  two  epigrams  from  Emerson's  essays. 

9.  Write  a  paragraph  on  the  subject — Thoreau's  Individualism. 

10.  For  what  were  the  following  noted:  Walt  Whitman,  John  Motley,  Joel  Barlow,  Timo- 
thy Dwight,  Francis  Parkman,  Thomas  Bailey  Aldrich,  William  Dean  Howells,  Bret 
Harte,  Bayard  Taylor,  Sidney  Lanier. 

11.  Name  the  authors  of  the  following:  Tales  of  a  Wayside  Inn,  Commemoration  Ode, 
The  Prairie,  The  Prince  of  Parthia,  Laus  Deo,  My  Study  Windows,  The  Last  Leaf, 
Tampa  Robins,  The  Blithedale  Romance,  Early  Spring. 

CIVICS 

(Eighth  grade) 

Answer  ten  questions. 

1.  How  does  the  Child  Labor  law  govern  the  employment  of  children  in  Illinois? 

2.  What  provisions  are  found  in  the  U.  S.  Constitution  in  regard  to  the  right  to  vote? 

3.  What  are  the  voting  qualifications  in  Illinois? 

4.  Explain  how  the  President  of  the  U.  S.  is  elected. 

5.  How  does  a  postman  secure  his  position?     What  are  some  of  the  necessary  qualifica- 
tions? 

6.  Write  the  Preamble  to  the  Constitution  of  the  U.  S. 

7.  In  what  particulars  were  the  Articles  of  Confederation  faulty? 

8.  When  did  the  Constitution  of  the  U.  S.  go  into  operation? 

9.  What  is  the  purpose  of  a  writ  of  Habeas  Corpus? 

10.  How  may  a  bill  be  passed  over  the  President's  veto? 

11.  State  the  duties  of  the  County  sheriff;  the  County  Superintendent 'of  Schools. 

12.  How  many  directors  are  there  in  school  districts  of  less  than  1,000  inhabitants? 

13.  What  constitutes  the  Illinois  Teachers'  Examining  Board?    What  are  its  duties? 

14.  State  briefly  the  duties  of  the  County  Clerk. 

15.  What  is  minority  or  proportional  representation?     How  is  it  used  in  Illinois? 

CIVICS 

(Eighth  grade) 

Answer  ten  questions: 

1.  What  is  a  democracy?     Name  two.     Compare  our  government  to  a  ball  team;  explain 
an  aristocracy  through  a  ball  team. 

2.  What  does  majority  rule  mean?    Was  it  right  for  us  to  resist  Britain  in  1775?    Why? 
Is  a  revolution  ever  dangerous? 

3.  Name  five  rights  of  American  citizens.     Name  five  duties  of  American  citizens. 
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4.  Where  did  we  get  our  ideas  of  liberty?     What  was  the  Magna  Charta? 

5.  Explain  home  rule  in  the  United  States.     W7ho  was  responsible  for  the  good  or  bad 
government? 

6.  Name  the  three  branches  of  our  government,  and  the  representative  of  each. 

7.  Who  may  become  president?     What  great  law  tells  us  this?    W7ho  is  commander-in- 
chief  of  the  army  and  navy? 

8.  When  and  where  did  the  Constitution  Convention  meet?  W7ho  made  the  Constitution 
a  power? 

9.  Write  the  Preamble  to  the  Constitution. 

10.  Name  five  of  the  president's  secretaries  and  tell  who  fills  the  offices. 

11.  Who  makes  treaties  and  issues  passports?     Who  has  charge  of  the  mints?    What  is 
the  difference  between  civil  and  political  rights?     When  do  our  political  rights  begin? 

CIVICS 
(High  school,  1st  year) 

1.  What  is  an  "unwritten  constitution?"     Give  examples  to  show  that  we  have  one. 

2.  Name  our  colonial  possessions  and  tell  how  each  is  governed. 

3.  Give  qualifications  and  length  of  term  for  the  President,  a  Senator,  and  a  Representa- 
tive. 

4.  What  is  meant  by  gerrymander,  pocket  veto,  quorum  pacifist,  recall,  neutral,  arbitra- 
tion? 

5.  Trace  a  bill  through  the  process  of  becoming  a  law. 

6.  Of  what  does  the  Supreme  Court  consist?  and  what  are  its  duties? 

CIVICS 

(High  school) 

Answer  eight  questions. 

1.  What  differences  did  the  framers  of  the  constitution  intend  to  create  in  the  two  houses? 

2.  What  dangers  are  inherent  in  popular  government? 

3.  "The  federal  government  is  one  of  limited  power  but  within  its  own  field  it  is  supreme." 
Discuss. 

4.  What  are  the  functions  of  the  grand  jury?  Of  the  petit  jury?  W7hat  is  a  "hung" 
jury? 

5.  Discuss  the  origin  of  political  parties  in  the  United  States.  State  the  forces  at  work  and 
the  political  leaders.  Show  how  the  different  political  parties,  when  in  power,  affect 
commerce  and  general  prosperity  of  the  nation. 

6.  Should  we  abandon  our  present  electoral  system?     Give  your  reasons. 

7.  What  is  the  difference  between  obeying  a  law  and  obeying  a  person? 
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8.  Discuss  metallic  and  paper  money  in  the  United  States,  stating  the  backing  of  each. 
Explain  the  European  money  market  today  on  the  basis  of  the  above  explanation. 

9.  What  is  the  work  of  the  National  Committee? 

10.     What  has  the  Washington  Conference  really  accomplished? 


GEOGRAPHY 

(Seventh  grade) 

1.  (a)     Describe  the  formation  of  our  continent. 

(b)     Name  the  two  great  mountain  systems  and  their  smaller  groups. 

2.  (a)     Describe  the  size,  shape  and  position  of  North  America, 
(b)     Wrhat  was  the  extent  of  the  Great  Ice  Sheet? 

3.  (a)     Write  an  interesting  paragraph  about  the  Eskimos, 
(b)     Name  the  New  England  States  and  give  their  capitals. 

4.  Describe  the  surface  features,  climate,  rainfall  and  products  of  the  Middle  Atlantic 
States. 

5.  (a)     Describe  the  mining  of  coal  in  the  Middle  Atlantic  States, 
(b)     Name  the  by-products  of  petroleum. 

6.  Name  and  describe  the  three  chief  industries  of  the  Southern  States. 


PHYSICS 
(High  school) 

Answer  ten  questions. 

1.  What  is  light?     Cause  of  eclipse  of  moon?     Show  by  drawing.     Draw  eclipse  of  sun. 

2.  Describe  the  rainbow.     Show  the  formation  in  drawing.     Show  by  drawing  why  the 
bow  is  curved. 

3.  What  is  heat?     Temperature?     WThy  does  the  boiling  point  vary? 

4.  What  is  the  heat  of  fusion?     Vaporization?     How  do  these  facts  affect  our  life? 

5.  Describe  a  heating  plant.     Draw.     (Either  steam,  water  or  air). 

6.  What  is  Probable  Nature  of  Electrification?     Theory  of  Lyden  jar? 

7.  Describe  a  good  open  circuit  battery  and  a  good  closed  circuit  battery. 

8.  Describe  what  you  saw  in  X-ray. 

9.  How  does  a  motor  work?     Describe  the  arc  light.     Incandescent. 

10.  Explain  induction  coil.     Give  practical  uses  of  the  coil.     Give  uses  of  transformer. 

11.  Describe  either  phone  or  telegraph  in  full. 

12.  Write  8  points  either  in  favor  or  against  the  study  of  physics  in  high  school. 
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ZOOLOGY 

(High  school) 

Answer  ten  questions. 

1.  Name  at  least  eight  branches  of  the  animal  kingdom  and  an  example  of  each  class. 

2.  Discuss  the  classification  of  animals  as  to  method  used  and  basis  for. 

3.  Explain  how  the  amphibia  stand  between  the  fishes  and  the  reptiles. 

4.  Give  four  illustrations  showing  how  insects  are  adapted  to  their  environment. 

5.  Why  are  the  porifera  a  step  higher  than  the  protozoa? 

6.  Name  an  animal  possessing  one  of  the  following: 

1.  Alternation  of  generation.  3.     Complete  metamorphoses 

2.  Bilateral  symmetry.  4.     Budding. 

5.     Regeneration  of  lost  parts. 

7.  Give  an  example  to  illustrate  the  struggle  for  existence  and  tell  how  the  law  of  the  sur- 
vival of  the  fittest  came  to  be  established. 

8.  Why  are  the  primates  of  such  great  importance? 

9.  What  animal  would  you  prefer  to  watch  and  study?     Why? 

10.  Explain  how  Zoology  helps  you  to  realize  the  following  objectives:     1.     Health. 
2.     Vocation.         3.     Use  of  leisure  time. 

11.  What  is  your  opinion  as  to  the  Theory  of  Evolution? 

ARITHMETIC 

(Eight  grade) 

Answer  ten  questions. 

1.  How  much  must  I  pay  for  U.  S.  4)4%  Liberty  Bonds  at  92,  brokerage  1}4,  in  order  to 
have  an  annual  income  of  $600? 

2.  How  many  blotters  6  inches  long  and  3}4  inches  wide  can  be  cut  without  waste  from  a 
sheet  of  blotting  paper  2  feet  long  and  14  inches  wide? 

3.  What  mathematical  facts  do  the  following  numbers  represent:     231,  7.92,  7000,  5280, 
360,  62K,  31^,32,  60,  16. 

4.  A  baseball  diamond,  or  infield,  of  regulation  size  for  men  is  90  ft.  square.     How  long  is  a 
straight  throw  from  first  base  to  third? 

5.  Extract  the  cube  root  of  5832  and  148877. 

6.  Draw  a  figure  to  represent  a  section  of  land,     (a)  Number  correctly  the  sections,     (b) 
In  a  smaller  drawing  show  the  N.  E.  *4  of  the  S.  E.  J4  of  Sec.  16. 

7.  Define  mensuration,  plane  surface,  rectangle,  trapezoid,  parallelogram. 

8.  How  find  the  area  of  a  parallelogram?     How  many  square  feet  in  a  building  lot  125  feet 
long  and  50  ft.  wide? 

9.  How  many  acres  of  land  in  a  road  10  mi.  long  and  4  rods  wide?     What  is  the  land  in 
this  road  worth  if  land  sells  at  $300  per  acre? 
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10.     Define  circle,  diameter,  radius,  circumference. 

12.  A  boy  measured  the  distance  around  a  tree  and  found  it  to  be  6%  ft.     How  thick  is 
the  tree,  correct  to  the  nearest  inch,  where  he  made  the  measurement? 

13.  What  is  the  lateral  surface  of  a  cylinder  15  inches  high  and  10  inches  in  diameter?    What 
is  its  entire  surface? 

14.  A  silo  (cylindrical)  is  12  ft.  in  diameter  and  is  filled  to  a  depth  of  18  ft.     How  many 
cubic  feet  of  silage  does  it  contain? 

15.  Define  sphere,  area  of  sphere,  volume  of  a  sphere.     Considering  the  earth  a  sphere  whose 
radius  is  4000  mi.,  find  the  area  of  the  earth's  surface.     Its  volume. 
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